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Abstract. We study combinatorial aspects of the Schubert calculus of the 
affine Grassmannian Gr associated with SL{n,C). Our main results are: 

• Picri rules for the Schubert bases of H* (Gr) and /f, (Gr), which expresses 
the product of a special Schubert class and an arbitrary Schubert class 
in terms of Schubert classes. 

• A new combinatorial definition for fc-Schur functions, which represent 
the Schubert basis of -ff*(Gr). 

• A combinatorial interpretation of the pairing if*(Gr) X if«(Gr) —* Z 
induced by the cap product. 

These results are obtained by interpreting the Schubert bases of Gr combina- 
torially as generating functions of objects we call strong and weak tableaux, 
which are respectively defined using the strong and weak orders on the affine 
symmetric group. We define a bijection called afiine insertion, generalizing 
the Robinson-Schensted Knuth correspondence, which sends certain biwords 
to pairs of tableaux of the same shape, one strong and one weaJc. Affine inser- 
tion offers a duality between the weak and strong orders which does not seem 
to have been noticed previously. 

Our cohomology Fieri rule conjecturally extends to the affine fiag mani- 
fold, and we give a series of related combinatorial conjectures. 
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Introduction 



Let Gr = G{C{{t)))/G{C[[t]]) denote the affine Grassmannian of G = SL{n,C), 
where C[[t]] is the ring of formal power series and C((i)) = C[[t]] [i^-*^] is the ring 
of formal Laurent series. Since Gr = Q/P for an affine Kac-Moody group Q and a 
maximal parabolic subgroup V, we may talk about the Schubert bases 

in the cohomology and homology of Gr, where 5" is the subset of the affine sym- 
metric group Sn consisting of the affine Grassmannian elements, which by definition 
are the elements of minimal length in their cosets in Sn/ Sn- Quillen (unpublished), 
and Garland and Raghunathan [6] showed that Gr is homotopy- equivalent to the 
group VlSU{n,'C) of based loops into SU{n,C), and thus iJ*(Gr) and i7*(Gr) ac- 
quire structures of dual Hopf- algebras. In ^3^, Bott calculated H*{Gr) and i7*(Gr) 
explicitly - they can be identified with a quotient A'"^ and a subring A(„) of the ring 
A of symmetric functions. Using an algebraic construction known as the nilHecke 
ring, Kostant and Kumar [13] studied the Schubert bases of H*{Gi) (in fact for 
flag varieties of Kac-Moody groups) and Peterson |32| studied the Schubert bases 
of i/*(Gr). Lam [16j . confirming a conjecture of Shimozono, identified the Schubert 
classes ^'^ and explicitly as symmetric functions in A'"' and A(„). 

In cohomology, the Schubert classes are given by the dual k-Schur functions 

{F^ I w e 5,"J c A("), 

introduced in |22) by Lapointe and Morse. The dual fc-Schur functions are gen- 
erating functions of objects called fc-tableaux |20| . In |15| . this construction was 
generalized to the case of an arbitrary affine permutation w G Sn] fc-tableaux are re- 
placed by what we call weak tableaux and the generating function of weak tableaux 
U is called the affine Stanley symmetric function or weak Schur function 

F^{x) = Weak^(x) = ^a;^'(^). 

u 

When w E Sn C Sn is a usual Grassmannian permutation (minimal length coset 
representative in Sn/{Sr x Sn-r)), the affine Stanley symmetric function reduces 
to a usual Schur function. 

In homology, the Schubert classes are given by the k-Schur functions 

{s[^\x) I Ai <n}CA(„) 

vii 
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where the k in fc-Schur or dual fc-Schur function always means 

k = n — 1. 

The /c-Schur functions were first introduced by Lapointe, Lascoux, and Morse |18j 
for the study of Macdonald polynomials [29] . though so far a direct connection 
between Macdonald polynomials and the affine Grassmannian has yet to be estab- 
lished. A number of conjecturally equivalent definitions of fc-Schur functions have 
been presented (see [181,1191 1211122] V In this article, a fc-Schur function will always 
refer to the definition of [2H,l22j and we can thus view {s^x \^) | Ai < n = fc + 1} as 
the basis of A(„) dual to the basis {F^ | w G 5°} of A'-"' with respect to a bijection 

{A I Ai < n} S° 

(0.1) 

X t-^ w 

(see Proposition 18 . ISp . Given an interval [v,w] in the strong order (Bruhat order) 
on Sn, we introduce the notion of a strong tableau T of shape w/v; it is a certain 
kind of labeled chain from w to w in the strong order. We define a strong Schur 
function to be the generating function of strong tableaux T of shape w/v: 

Strong„/„(x)=^x-*(^). 

T 

One of our main results (Theorem 14. lip is that fc-Schur functions are special cases 
of strong Schur functions: 

Sx\x) = Strong„/id(a;) 

where id G Sn is the identity and A i-^ w e 5° under the bijection (jO.ip . When 
v^w e Sn C Sn are usual Grassmannian permutations, strong Schur functions 
reduce to usual skew Schur functions. Strong tableaux, in the case of 5„ C Sn 1 are 
closely related to chains in the fc-Bruhat order (where here fc is unrelated to n) of 
Bergeron and Sottile [l]. An important difference is that our chains are marked, 
reflecting the fact that affine Chevalley coefficients are not multiplicity-free (see 
Remark [2^ . 

Our main result (Theorem l4.2p is the construction of an algorithmically defined 
bijection called affine insertion. In its simplest case, affine insertion establishes a 
bijection between nonnegative integer matrices with row sums less than n, and pairs 
(P, Q) where P is a strong tableau, Q is a weak tableau, and both tableaux start 
at id S Sn and end at the same w S 5". This bijection reduces to the usual row- 
insertion Robinson-Schensted-Knuth (RSK) algorithm (see [5]) as rt — > oo. Affine 
insertion yields a combinatorial proof of the following affine Cauchy identity, which 
is obtained from Theorem 14. 41 by taking u = v = id: 

^n{x,y)= ^ strong^ (a;) Weak„(2;) 
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and the affine Cauchy kernel is given by 



^n{x,y) ='[[{^ + y^hl{x)+yfh2{x) + ■■■ + y'; ^hn-i{x)) 



= X! hx{x)mx{y). 



A : Ai <n 



This provides a combinatorial description of the reproducing kernel of the per- 
fect pairing 



induced by the cap product. 

The bijection also yields geometric information in the form of Fieri rules, which 
are explicit formulae for certain structure constants with respect to the Schubert 
bases of iJ*(Gr) and i7*(Gr). We use affine insertion to derive the Fieri rules 
fTheorem 14.71 and 14. 9p for strong and weak Schur functions. 

Affine insertion exhibits a duality between the weak and strong orders which 
does not seem to have been studied before, even in the case of the finite symmetric 
group Sn- In particular one may show that the number of pairs (P, Q) of a standard 
strong tableau and a standard weak tableau of the same "shape" (starting at the 
identity id and ending at some permutation i/; G S*^) and size m is given by m!. 
These issues are pursued in |17j where such identities are established in Kac-Moody 
generality. One may also generalize this duality in a different direction, to obtain 
similar identities for combinatorial Hopf algebras. 

The construction and proof of the affine insertion algorithm is reduced to a 
"local rule" using the technology of Fomin's growth diagrams [4j. The local rule, 
which is constructed directly on the level of affine permutations, represents the 
most involved part of this paper. Our local rule has many elements which will 
be familiar to experts of Schensted insertion, including local bijections analogous 
to boxes being bumped to the next row, or boxes not interfering with each other. 
The strong covers a; < y of in this article roughly correspond to boxes in the 
traditional language of Young tableaux. 

As corollaries of the affine insertion theorem, one deduces Fieri rules for the 
i/*(Gr) and i/*(Gr). Let co,™ = Sm-i • • • sisa G Sn where {si\i € 'L/n'L} denote 
the simple generators of Sn- We obtain an affine homology Fieri rule in H^^,{Gv) 



where the sum runs over weak strips w z of size m and an afRne cohomology 
Fieri rule for _ff*(Gr) (Theorem |4T3]) : 



where the sum runs over strong strips S of size m with inside(S') — w. The affine 
cohomology Fieri rule is an affine analogue of the usual Fieri rule for the Schubert 
calculus of the flag manifold [^7\ . The affine homology Fieri rule does not appear 



iJ*(Gr) X i/,(Gr) 



(Theorem ETH): 




■W 




S 
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to have a classical geometric counterpart. It can be deduced directly from the 
identification of the afhne homology Schubert basis \ w G S""} as fc-Schur 
functions in [16] . together with the Fieri rule for A:-Schur functions first stated 
in [21] and described in the notation of this article in [23j . 

Conj ecture 14 . 1 51 assert s that the natural analogue of the affine cohomology Fieri 
rule holds for the cohomology H*{Q/B) of the affinc flag variety. This conjecture 
is related to a series of combinatorial conjectures (Conjecture I4.18P concerning 
the strong Schur functions Strong^^^ (x) , including symmetry and positivity when 
expressed in terms of fc-Schur functions. The analogous properties for weak Schur 
functions were established in |15^ 116) . 

In the last part of our paper we translate weak and strong tableaux, together 
with the affine insertion bijection, from permutations into the more traditional 
language of partitions. This is performed using a classical bijection [241130] between 
5"^ and the set of partitions which are n- cores. For weak strips and weak tableaux, 
the corresponding combinatorics involving cores was worked out in j20) . Our main 
result here (Froposition lQ . 5| l gives a purely partition-theoretic description of marked 
strong covers, and hence strong strips and strong tableaux. As a consequence the 
affine cohomology Fieri rule acquires a form similar to that of the Fieri rule for Schur 
functions, with horizontal strips replaced by "strong strips built on cores". We also 
use the combinatorial description of strong covers to define a "spin" -statistic on 
strong tableaux and conjecture (Coniecture l9.1l|l that the original A:-Schur functions 
(depending on a parameter t) of [181 119] are spin-weight generating functions of 
strong tableaux of fixed shape. 

Our work poses further challenges for both geometers and combinatorialists. 
The two Fieri rules beg for a more geometric proof; in the cohomology case there 
should be a geometric proof similar to that of Sottile [37" , and alternatively a more 
algebraic derivation might be possible using the recursive machinery of Kostant 
and Kumar's nilHecke ring [13|. The "monomial positivity" of the cohomology 
classes ^"^ — Weak„(a;) can be interpreted geometrically as arising from Bott's 
map (CP")°° Gr. It would be interesting to obtain a geometric explanation of 
the monomial positivity of — Strong^^ (x) . 

The positivity of structure coefficients for both weak and strong Schur functions 
have yet to be given a combinatorial interpretation. The structure constants for 
the strong Schur functions yields as a special case the WZW fusion coefficients 
(or equivalently the structure constants of the quantum cohomology QH*{G/P) 
of the Grassmannian) as proved by Lapointe and Morse [22j . More generally, 
Feterson has shown that the structure constants of the quantum cohomology of any 
(partial) flag manifold can be obtained from the structure constants of the homology 
i/*(Gr) of the affine Grassmannian. Obtaining a combinatorial interpretation for 
these structure constants is likely to be a challenging problem. Another interesting 
problem is to give a direct combinatorial proof of the symmetry of strong Schur 
functions. 

The literature contains other "affine" or "infinite" generalizations of the Schen- 
sted algorithm, see for example [311, 134] . However, as far as we are aware, these 
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algorithms biject afRnc or infinite permutations with certain tableaux, while our 
insertion pairs usual permutations with affine tableaux. 

Organization. This paper is roughly divided into three parts. In the first part 
(Chapters [iHll) we give the necessary definitions and present our main theorems. 
Chapter[l]contains notation for symmetric functions and Schubert bases of the affine 
Grassmannian. Our two main objects, strong and weak tableaux, are introduced in 
Chapters [2] and [3] respectively. In Chapter [H we present and prove our main results 
modulo the proof of affine insertion. The proof of the affine insertion algorithm is 
reduced to properties of local rules using Fomin's growth diagram machinery. 

The local rules for the affine insertion algorithm are defined and studied in the 
second, and most technical, part of our paper (Chapters [SHT]). In Chapters [5] and [6] 
we define the forward and reverse local rules, and show that they are well-defined. 
In Chapter [7] we prove that affine insertion is bijective. 

The last part of our paper (Chapters [5 HTU|) contains translations of our combi- 
natorial constructions into the language of partitions and cores. In Chapter [8l we 
explain a number of bijections between the coroot lattice, affine Grassmannian per- 
mutations, cores, offset sequences and fc-boundcd partitions. In Chapters [9l and fTOl 
we explain weak and strong tableaux and affine insertion using the combinatorial 
language of cores. 



CHAPTER 1 



Schubert Bases of Gr and Symmetric Functions 

1.1. Symmetric functions 

Here we introduce notation for symmetric functions, wiiicii can be found in 
greater detail in [29] . Let A = A{x) denote the ring of symmetric functions in in- 
finitely many variables xi,X2, ■ ■ ■ over Z. It is generated over Z by the algebraically 
independent homogeneous symmetric functions hi,h2,---, where deghi = i. The 
ring A is equipped with an algebra involution a; : A — > A given by uj{hi) ~ where 
Ci denotes the elementary symmetric Junctions. For a partition A — (Ai > A2 > ■ ■ ■ ) 
let h\ := h\-^h\2 ■ ■ ■ . The Hall inner product ( , )a : A x A ^ Z is defined by 
{h\ , to^)a — Sxf^ where denotes the monomial symmetric functions. 

The ring A has a coproduct A : A — » A^zA given by A{hi) = J2o<j<i hj®hi-j 
where Hq := 1. Together with the Hall inner product, this gives A the structure of a 
self-dual commutative and cocommutative Hopf algebra. The antipode c is given by 
c{hi) = {—lyei] thus if / G A is homogeneous of degree d then c(/) = (— l)''a;(/). 

Now let A^"^ A/{m\ \ Ai > n). This is a quotient Hopf algebra of A. Let 
A(„) := Z[/ii, ft,2, . . . , /in-i]. This is a sub-Hopf algebra of A. The Hall inner product 
gives A*^"^ and Ajn) the structures of dual Hopf algebras. One possible choice of 
dual bases is {mx \ Ai < n — 1} for A^") and {h\ \ Ai < n—1} for A(„). The algebra 
involution w of A restricts to an involution of A(„-). By duality we also obtain an 
involution uj^ of A'"^ characterized by the property (/ , g)A — {uj{f) , w+(5))a for 
/ G A(„) and g G A^"'. For / G A let / G A^") denote its image in the quotient. 
Then = (^{f)- If / G A, when the context makes it clear we will just write / 

to denote its image in 



1.2. Schubert bases of Gr 

Let Gr denote the affine Grassmannian of SL{n,C). It is an ind-scheme 
equipped with a Schubert-decomposition 

Gr= □ U 

where the unions are taken over the set S"^ of all 0- Grassmannian permutations 
in the affine symmetric group Sn (see Chapter [2|) ; fi^ denotes the Schubert cell 
indexed by w and denotes the Schubert variety. Let S,^ G H*{Gr) and ^u, G 
i/*(Gr) denote the corresponding Schubert classes in cohomology and homology; 
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1. SCHUBERT BASES OF Gr AND SYMMETRIC FUNCTIONS 



see [11 116^ I14j . The cap product yields a pairing 

(■, ■)Gr:i?*(Gr)x77,(Gr)^Z 

under which the Schubert bases {^"^ \ w £ S^} and {£_w \ w E 5*°} are dual. 
Throughout this paper, all (co)homology rings have coefhcients in Z. For u,v,w E 
define c^^ E Z and d^, G Z by 

w 

(1.2) rr = E^™?"'- 

The structure constants c^y were studied in [13j using the nilHecke ring. It follows 
from work of Graham fS] and Kumar [14] that c™^ G Z>o and from work of Peter- 
son [32] that d™^ E Z>o . Our work yields combinatorial interpretations for some of 
these numbers. 

The space Gr is homotopy-equivalent to the based loop space flSU (n) ; see [6l 
133] . Thus ff*(Gr) and H*{Gr) are endowed with the structures of dual commu- 
tative and co-commutative Hopf algebras. In ^3 , Bott calculated these Hopf alge- 
bras explicitly. By identifying the generators explicitly one obtains isomorphisms 
H*{Gr) = A(") and i?*(Gr) = A(„) such that the diagram 

H*{Gr) X iJ,(Gr) 




A(") X A(„) 

commutes. 

A natural problem is the identification of the Schubert classes ^u, and as 
symmetric functions. Confirming a conjecture of Shimozono, in [16j Lam showed 
that the Schubert classes and are represented respectively by the k-Schur 
functions [181 119L I21j and affine Schur functions (also called dual k-Schur func- 
tions) [H] [22] . 

Theorem 1.1. TF Under the isomorphism H*{Gr) = A^"), the Schubert class 
^'^ is sent to the affine Schur function Fyj E A*^"'' . Under the isomorphism (Gr) = 

(k) 

A(„), the Schubert class is sent to the k-Schur function Sw E A(„), where 
k — n — 1. 

The affine Schur functions Fu, are generating functions of combinatorial objects 
known as k-tableaux and were first introduced by Lapointe and Morse in [20] . We 
shall define these objects in Chapter [31 following the approach of [15j . It is shown 
in [m [22] that {F^ \ w E S°} forms a basis of A("). The fc-Schur functions 
{sif^ \ w E S'^} form the dual basis of A(„) to the affine Schur functions. The 
/c-Schur functions s^^\x) or Sw\x) used here are conjecturally (see [21] ) the t = 1 
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specializations of the fc-Schur functions s]^ {x;t) first introduced by Lascoux, La- 
pointe, and Morse in [18] to study Macdonald polynomials. The fc-Scliur functions 
(and affine Schur functions) are usually indexed by partitions A such that Ai < k. 
We will describe the bijection between the sets {A | Ai < k} and {w S S*"} in 
Chapter [51 For the first portion of this paper we will use affine permutations as 
indices for affine Schur and fc-Schur functions. 

1.3. Schubert basis of the afRne flag variety 

Let Q/B denote the flag variety for the affine Kac-Moody group SL(n,C). 
Again we omit the construction of this ind-schcmc and refer the reader to [14| . The 
space G/B has a decomposition into Schubert varieties indexed by affine permuta- 
tions We let e H*{Q/B) for w € Sn denote the cohomology Schubert 
basis of the affine flag variety. There is a (surjective) morphism Q /B ^ Gx which 
induces an algebra inclusion t : iJ*(Gr) ^ H*{Q/B). The Schubert classes are sent 
to Schubert classes under t, so that = for w S 5°. Thus we may define 
integers c™^ G Z as the structure constants of H* (Q/B): 

(1-3) CbCb-^T.<^^b 

w 

and when w,u,v G S'^ this agrees with the definition in Section 11.21 Again, by 
general results of [HI [14], the integers c™^ are nonnegative. 



CHAPTER 2 



Strong Tableaux 



We introduce some combinatorial constructions involving the affine symmetric 
group Sn. For a £ Z let a denote the coset a + nL e 'L/n'L. 

2.1. S*,! as a Coxeter group 

The affine symmetric group Sn is an infinite Coxeter group, with generators 
{soj Si, ... , s„_i} of simple reflections and relations sf = id, SiSj — SjSi if i and 
j are not adjacent in Z/nZ, and SiSjSi = SjSiSj if i and j are adjacent in Z/nZ. 
Here pairs of elements of the form {z, i + 1} are adjacent in Z/nZ and other pairs of 
elements are not. The length £(w) of w £ Sn is the number £ of simple reflections in 
a reduced decomposition oiw, which by definition is a factorization w = Si-^Si^ ■ ■ ■ Si^ 
of w into a minimum number of simple reflections. 

For a,b G Z we write Sa — si, ii a — h. A reflection is an element that is 
conjugate to a simple reflection. 

2.1.1. 5,1 as periodic permutations of Z. The affine symmetric group Sn 
can be realized as the set of permutations w of Z such that wii + n) = w{i) + n 
for all i € Z and J27=ii''^(^) ^0 = (see t2Sj)- We sometimes specify an element 
w € Sn hy "window" notation 

w = [w(l),w(2), . . .,w{n)] 

as this uniquely determines w. Multiplication of elements v,w € Sn is given by 
function composition: {vw){i) = v{w{i)) for all i G Z. We recall Shi's length 
formula [34] 

(2.1) iM- E ^^^^^ 

l<i<j<n 

An inversion of w £ Sn is a pair (i, j) G Z^ such that i^j,i<j,l<i<n, and 
> Let Inv(ti;) denote the set of inversions of w. Then we have (see |10j ) 

(2.2) l{w) = |Inv(w)|. 

For r, s ^Z with r 7^ s let tr,s be the unique element of Sn defined by i,.^s ('') = s, 
ir,s(s) = r, and tr,s{i) = i for « ^ {^js}- We have 

(2.3) wtijW~^ = for aU w G 

The simple reflections are given by Si — ii.i+i for z G Z. By (|2.3|) the reflections in 
S"™ are precisely the elements of the form tr^s. 

5 
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2. STRONG TABLEAUX 



Example 2.1. For n = 3, io,4 = [-3, 2, 7] since ^0,4(1) = to,4(4)-3 = 0-3 = -3 
and io.4(3) = to,4(0) + 3 = 4 + 3 = 7. 

One may also specify w G Sn in "two-line notation" by expanding the window 
to include all values of w : Z ^ Z: 

-2 -1 1 2 
••• w{-2) w{-l) w{0) w{l) w{2) ■■■ 

In this notation ty w is obtained from w by the exchanging the values i + kn and 
j + kn in the lower row for all fc G Z. By (|2.3p the permutation w tij is obtained 
from w by exchanging the elements in the positions i + kn and j + fcrt for all fc G Z. 

2.2. Fixing a maximal parabolic 

For the duration of the paper, fix / G Z. Let 5^ C 5,1 be the maximal parabolic 
subgroup generated by Si for i j^l. It is isomorphic to 5„ . The definitions of strong 
strip and strong tableau in Section [^751 depend on the choice of /. We say that an 
element w G Sn is l-Grassmannian if it is of minimum length in its coset in Sn/S^, 
that is, w{l + 1) < w{l + 2) < • • • < w{l + n). We say that w is Grassmannian to 
mean that it is 0-Grassmannian. 

2.3. Strong order and strong tableaux 

The strong order on Sn is by definition the partial order with covering relation 
w < u, which holds exactly when £{u) = £{w) + 1 and wtij = u for some reflection 
tij. 

For a, 6 G Z we sometimes use the notation [a,b] = {c E Z, \ a < c < b} and 
{a, b) — {c E Z \ a < c < b} for intervals of integers. 

Lemma 2.2. Let w G Sn and let i < j be integers such that i ^ j. Then 

(1) w < wtij if and only if w{i) < and for each k G w{k) ^ 
[w{i),w{j)]. 

(2) wtij < w only if w{j) < w{i) and for each k G (i,j), w{k) ^ [w{j),w{i)]. 
Moreover if the strong cover holds then either j — i < n or \w{i) ~ w{j)\ < n. 

Proof. Suppose w{i) < w{j). We have 

tij — (SiSi+l • • • Si+a-l)Si+a{Si+a-l ' ' ' Sj+lSj) 

for i + a = j — 1. Let 

= w SiSi+i • • • for r G [0, a + 1] 

W(r) = w'^"^'^'' Si+a-1 ■ ■ ■ Si+a-r for r G [0, a]. 

We have ^(w^) = ± 1 for r G [l,a + 1] and = ^(w(r-i)) ± 1 

for r G [I, a]. Note that in passing from w = w'^"^ to w*^") (resp. w^^+i) = wj-q) to 
W(a) = wtij), given A, B and C as defined below, we compare w{i) (resp. w{i)) with 
all ^(fc) where k E A\JB (resp. fc G ^UC) and count the number of inversions that 
have been gained or lost. Note that w{j) > w{i) > w{k) for all k E C. Including 
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the "middle" step w*^"^ to that exchanges the elements w{i) and w{j) in 

adjacent positions j ~ 1 and j respectively, and writing 

A={ke |i^{IJ}} 
^ {k E A \ w{k) < w{i)} 
A+ = {ke A \ w{k) > w{j)} 
A= ^ {k e A \ w{i) < w{k) < w{j)} 

B = {keii,j)\k^3} 
B+ = {kE B \ w{k) > w{i)} 
B^ = {kEB\ w{k) < w{i)} 

C — {i — n,i — 2n, . . . ,i — \ (,w{j) — w(i)) j n\n} , 

we have 

= + + 1^+1 - 1^-1 + \B+\ - + 1; 

l{wU,) = i{w^a)) - l{w^''+'^) + \A=\ - \A+\ + + |C|. 

But |B| = \B+\ + \B_\ = \C\. So t{wtij) ~ £{w) = 2\A^ + 2\B+\ + 1. Therefore 
w < wtij if and only if — and \B+\ — 0. But the following are equivalent: 
5+ is empty; j — i < n ot w{j) — w{i) < n; for every k E (i, j) with k E {i, j}, w(fc) 
is not in the interval {w{i),'w{j)). Part (1) follows. 

The statement in Part (2) follows immediately from Part (1). □ 

Example 2.3. Even if w{j) < w{i) and for each k E we have w{k) ^ 
[w{j),w{i)], it is not always true that wtij < w- For example, let n = 3, w = 
[10, 2, —6], i — 1, j = 5. Then w is not a strong cover of wtij — [5, 7, —6]. 

A marked strong cover C = (w u) consists of w, u E Sn and an ordered 
pair E 1? such that 

(1) w < M is a strong cover with w tij = u. 

(2) The reflection tij straddles I, that is, i < / < j, where I is defined in 
Section O 

We use the notation inside(C) = w and outside(C) = u. We say that C is marked 
at the integer 

(2.4) m(C) = w{j) = u{i). 

Remark 2.4. Let w < u he a strong cover. The number of pairs such 
that w u is a marked strong cover, is equal to the affine Chevalley multiplicity, 
which by definition is the structure constant for the Schubert basis of the 

cohomology H*{Q /B) of the affine flag variety. This is merely a translation of the 
Chevalley rule in [13] for a Kac-Moody flag manifold, in the special case of the 
affine flag variety. 

Remark 2.5. Our notion of strong marked cover when restricted to the sym- 
metric group Sn C Sn essentially produces the fc-Bruhat order studied by Sot- 
tile [37] and Ber geron and Sottile [1]. 
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Proposition 2.6. // w — ^- u is a marked strong cover and w is l-Grass- 
mannian then u is l-Grassmannian as well. 

Proof. Suppose w is ^-Grassmannian so that w{l + 1) < w{l + 2) < • • • < 
w{l + n). We may pick i\j' so that t^j = ti/jv and i' < I < j' < I + n. By 
Lemma we must have w{i') < wij') and for each k satisfying Z + 1 < fc < j' we 
must have 'w{k) < w(i') (since w^k) < w{j')). In particular, since k > i' , we must 
have ky^l/. Thus u{l + 1) < u{l + 2) < ■ ■ ■ < u{j'). 

Now suppose i' = a where j' < a < I + n. Then u{a) > w{a) so we must 
have u{a) > u{a — 1) > • • • > Now for each k satisfying a < k < I + n, 

we have i' < k ~ bn < j' where i' — a — bn for a positive integer b. Thus u{a) = 
w(j') + hn. Again by Lemma [2.2i we have either w{k — bn) < w{i') — w{a) — bn 
or w{k — bn) > w{j'). Since z«(fc) > w{a) the first situation does not occur. Thus 
u{k) = w{k) > w{j') + bn = u{a). Combining the inequahties, we conclude that u 
is Z-Grassmannian. □ 

A strong tuple S = [w; (Ci, C2, . . . , Cr); u] consists of a "sequence of marked 
strong covers from w to w", that is, elements w,u d Sn and a sequence of marked 
strong covers Ck such that outside(CA;) — insidc(Cfc+i) for each < fc < r, where by 
convention outside(Co) = w and inside(Cr+i) = u. This is a certain kind of chain 
in the strong order from w to m with data attached to the covers. Sometimes w 
and u are suppressed in the notation. We write inside(S') = w and outside(S') — u. 
The size of S is the number r of covers in S. By definition size(S') = i{u) — £{w). 
If size(S') > 0, we refer to the first and last covers in S by first(5) = Ci and 
last(S') = Cr- If size(S') = then the empty strong tuple S is determined only 
by the element inside(S') = outside(5). We also use the notation C~ and to 
refer respectively to the cover before and after the cover C within a given strong 
tuple S = (. . . , C^, C, C+, . . . ). If C — {w u) then we use the notation 

— (w^ i u^) and C+ — (w+ 

A strong strip is a strong tuple S with increasing markings, that is, m(Ci) < 
m{C2) < ■ ■ ■ < m{Cr)- A strong strip of size 1 is the same thing as a strong cover. If 

we wish to emphasize the inside and outside permutations then we use the notation 

s 

w — > u. 

Lemma 2.7. Let S = (Ci, . . . , Cr) be a strong tuple. Then m(C]~) 7^ m{C]~+i) 
for 1 < k < r. 

Proof. Let = (. . . , C^, C, . . . ) and C = (w ^ u). We have m(C-) = 
u^{i^) = w{i^) and m{C) — w{j). So m{C^) = m{C) implies that i^ = j, which 
contradicts the straddling inequalities i^ < I < j. □ 

Given strong strips Si and ^2 such that outside(S'i) = insidc(52) let 5*1 U ^2 
denote the strong tuple obtained by concatenating the sequences of strong covers 
defining 5*1 and 6*2. Then S'l U S'2 is a strong strip if and only if one of the Si is 
empty, or if both are nonempty and m(last(S'i)) < m(first(52)). 
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A strong tableau is a sequence T ~ [Si, S2, ■ ■ ■) of strong strips Sk such that 
outside(S'fc) = inside(S'A;+i) for aU i G Z>o and size(S'fe) = for aU sufficiently large 
k. We define inside(T) — inside(S'i) and outside(r) = outside(S'/£) for k large. The 
weight wt(T) of T is the sequence 

wt(T) = (size(S'i),size(S'2 ),...). 

We say that T has shape u/v where u = outside(r) and v = inside(T). If T 
has shape u/id we simply say that T has shape u. 

2.4. Strong Schur functions 

Definition 2.8. For fixed u,w e Sn, define the strong Schur function 
(2.5) Strong„/„(x)=^x-'(^) 

T 

where T runs over the strong tableaux of shape u/v. 

We will use the convention that Strongj^ (x) — Strong„/jjj(a;). By Proposi- 
tion [2?6l Strongly.;; (x) = if w is /-Grassmannian and u is not. We shall show 
later in Theorem 14.111 that when u is /-Grassmannian, Strong„(a;) is a fc-Schur 
function and thus possess remarkable properties as shown in |21j . However, for 
general m, u G Sn, the generating function Strong^^^, (a;) is not well understood, es- 
pecially compared to the weak Schur functions to be introduced in Chapter [31 See 
Section I4?5l 

Example 2.9. Let u = ci^m ■= si+m-i • ■ • si+isi for an integer m satisfying 
< m < n — 1. For example, if I — then in window notation, u = [0, 1, . . . , to — 
1, TO + 1, TO -|- 2, . . . , n — 1, ri + to]. Let us calculate Strong„(x). The only strong 
cover C = (w u) where w is /-Grassmannian, is given by w = ci^m-i and 
{i,j) = (1,1 + w). This strong cover Cm is marked at m{C) ~ I + m. Since 
id G Sn is /-Grassmannian for any /, by Proposition l2 . 61 we see that a strong tableau 
T = {Si, S2, ■ ■ ■ , Sr) with shape u/id is determined by specifying integers = toq < 
mi < m2 < ■ ■ ■ <mr ^ m such that Sk = [c;,„i^_^ ; C™;,_i+i, ■ • ■ , C,„j^; c;,,„J. Thus 
Strong„(x) = hm{x). 



CHAPTER 3 



Weak Tableaux 

3.1. Cyclically decreasing permutations and weak tableaux 

This section follows jl5l , which builds on earlier work in the special case of 
O-Grassmannian elements (or n-cores) [201 122j . The intervals / = {a, a+ 1, . . . , 6 — 
1, 6} C Z/nZ considered in cyclical fashion will be denoted with the interval nota- 
tion [a, 6]. 

The left weak order ^ on Sn (sometimes also called the left weak Bruhat order) 
is defined by w ^ if and only if there is a u € Sn such that uw = v with 

£{u) + e{w) = e{v). 

Given a proper cyclic interval / — [a, b] C Z/nZ, let c/ = ShSb-i ■ ■ ■ Sa & Sn he 
the product of the reflections indexed by /, appearing in decreasing order. Given 
any proper subset A C Z/nZ let ca — cj-^ ■ ■ ■ c/^ where A = /i U • ■ • U is the 
decomposition of A into maximal cyclic intervals Ik which are called the cyclic 
components of A. The element ca is well-defined since ci- and c/^ commute for 
i J. Say that c € Sn is cyclically decreasing if c — ca for some A C Z/tiZ. Write 
A{c) for this subset A. 

Example 3.1. Let n = 10 and A = {0, 1, 3, 4, 6, 9}. The cyclic components of 
A are [9, 1], [3, 4] and [6, 6] and we have 

CA = isiSQSg){s4S3){S(i) = (S4S3) (si SQSg ) (se ) = • • • . 

Since 2 ^ A, the action of ca on Z acts on the "window" [3, 12] C Z by the cycles 
12 i-> 11 i-> 10 9 12, 5 4 3 i-> 5, 7 6 7, 8 8, and on aU of Z 
periodically. 

A weak strip W = {w --^ v) consists of a pair w, w G Sn such that w ^ v and 
vw~^ is cyclically decreasing; it is a certain kind of interval in the left weak order. 
If we wish to emphasize the cyclically decreasing element c = vw~^ then we write 

A 

W — {w ^ v) where A = A{c). The definition of ^ implies that £{c) +i{w) ~ i{v). 
The size of W, denoted size(Vl^), is the integer £{v) — £{w) = £{c) = \A\. We write 
inside(M^) — w and outside(W) = v. 

A weak tableau U is & sequence U ~ {Wi,W2, ■ ■■) of weak strips such that 
outside(Tyfe) = inside(Wfe-i-i) for all k e Z>o and size{Wk) = for A: large. Let 
inside(C/) — inside(Wi) and outside(J7) = outside(l¥fe) for k large. The tableau 
U gives precisely the data of a chain (insidc(Wi), inside(W2), • . . ) in the left weak 
order on Sn such that consecutive elements define a weak strip. The weight wt(U) 
of a weak tableau U is the composition wt{U) — (size(Wi), size(W2), . . . ). We say 
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that U is a tableau of shape u/v where u = outside(C/) and v = inside(i7). If U has 
shape u/id we say that U has shape u. 

Note that unHke strong tableaux, weak tableaux do not depend on the choice 
Z S Z of maximal parabolic. 

3.2. Weak Schur functions 

Definition 3.2. For m, u e Sn define the weak Schur funetion ot (skew) affine 
Stanley symmetric function by 

(3.1) Weak„/„(2;) = 

u 

where U runs over the weak tableaux of shape u/v. 

We use the shorthand Weak„(a;) to mean Weak^/j^ (x) . The generating func- 
tions Weak,(/^,(x) were first introduced in [15] where they were called skew affine 
Stanley symmetric functions, though weak tableaux were not explicitly used. It 
is not difficult to see that li v < u then Weak„/„(a;) = Weak„^-i (x). Thus each 
weak tableaux generating function is in fact an affine Stanley symmetric function 
(denoted Fi„(a:) in [15]). 

If w G S'n is 0-Grassmannian then weak tableaux of shape u are the k-tableaux 
(with k — n ~ 1) first defined in [201. We shall translate strong and weak tableaux 
into the context of fc-tableaux in Chapter [9l In the case that w G S*^, Weaktj(x) 
is the dual fc-Schur function introduced in [2 2) (called an affine Schur function 
in [HI). 

The basic theorem for Weak„/^(a;) is the following. 

Theorem 3.3 (Symmetry of weak Schurs ^15 ). For m, u G Sn, the generating 
function Weak„/^(x) is a symmetric function in Xi,X2, ■■ ■■ 

We shall later also need the following properties of Weak„/„ [x) . Let 

(3.2) w^w* 

be the unique involutive automorphism of the group Sn such that s* = si-i for all 
i. 

Theorem 3.4 (Conjugacy of weak Schurs ^15 ). Let w e Sn- Then 
aj+(Weak^(a;)) — Weaktu-i(2;) — Weakt„.(x). 

Theorem 3.5 (Affine Schurs form a basis [HI [22]). The set {Weaku,(x) | w e 
Sn} forms a basis o/A*^"^. 

Remark 3.6. If a denotes the map on Sn induced by a rotational automor- 
phism of the affine Dynkin diagram, then Weakt„(a;) — Weaker (x) (see also |15[ 
Propostion 18]). Thus 5'° can be replaced by Sn in Theorem 13.51 

Example 3.7. Let w = ci^m ■= si+m-isi+m-2 ■ ■ ■ si for m > a non-negative 
integer. Then w has a unique reduced decomposition. Note that c;.„i is always 
cyclically decreasing if m < n — 1 and that Q^mC; = Q+m'.m-m' if m > m' > 
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1. The weak tableaux U with shape w are of the form U = {Wi,W2, ■ ■ ■ ,Wr) 
where Wk = {ci,m.k-i ^ Q,™^) and 1 = mo < mi < m2 < ■ ■ ■ < ruk = m 
satisfies mt — mt-i < n — 1 for 1 < fc < r. The weight of U is wt{U) = (toi — 
mo,m2 — mi, . . . ,mr — mr-i). The weak tableau generating function, or affine 
Stanley symmetric function labeled by w is thus Weak^, (x) = F^, (x) , which is given 
by the image of hm{x) in A^"^. 



3.3. Properties of weak strips 

For later use we collect some properties of cyclically decreasing elements and 
weak strips. Let A C Ij/riL. We say that i € Z is A-nice if i — 1 ^ A and that i is 
A-bad iil ^ A. 

Given i e Z, let j be the smallest A-nice integer such that j > i. We have 



(3.3) i + n> CA{i) 



I j — 1 if i is A-nice 
[i — 1 otherwise. 

In other words, iii < j are consecutive A-nicc integers for A C Z/nZ, then j — i < n 
and CA acts on the set of integers [i,j — 1] by the cycle ^ — 1 j — 2 i-^ • • • i— > 
i + li-^ii-^j — 1 and i is the cyclic minimum of the cyclic component — 2] of 
A. In particular 

c^(i) = i — 1 if i is not A-nice 
(3.4) CA{i) = i if i is A-nice and A-bad 

CA{i) > i if i is A-nice and not A-bad. 

We obtain the following Lemmata. 

Lemma 3.8. Let A C Z/nZ. Then ca restricts to an order-preserving bijection 
from the set of A-nice integers to the set of A-bad integers. 

Lemma 3.9. Let A C Z/nZ and i < j. Then CA{i) > caU) if and only if 
j CA{i), i is A-nice and not A-had. Equivalently, CA{i) > c^(i) if and only if i 
is A-nice and the next A-nice integer after i is larger than j . In this case j — i < n 
and j is not A-nice. 

Lemma 3.10. Let c = ca be cyclically decreasing and v = cw. The following 
are equivalent: 

(1) w V is a weak strip. 

(2) For every pair of consecutive A-nice integers a <b we have 

w~'^{a) < min(«;"^(a + 1), w~'^{a + 2), . . .,w~'^{b - 1)). 

(3) For every pair of consecutive A-bad integers a <b we have 

v-\b) < min{v-\a + 1), . . . , u"^(6 - 1)). 

A 

Lemma 3.11. Let w v be a weak strip. Then there do not exist integers i < j 
such that w{i) > w{j) and v{i) < v{j). 
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Proof. Suppose i and j exist. By Lemma, \3l9\ w( j) is A-nice and w{j) < w{i) < 
b where b is the smaUest A-nice integer greater than Applying Lemma l3.10r 2) 

we have the contradiction j < i. □ 

Lemma 3.12. Let w v be a weak strip. If v is l-Grassmannian, then w is 
l-Grassmannian as well. 

Proof. Suppose ^(Z + l) < v{l + 2) < ■ ■ ■ < v{l + n). Then Lemma 13.1 II impUes 
that w{l + 1) < w{l + 2) < ■ ■ ■ < w{l + n). □ 

Lemma 3.13. Let A C TLjnL, q < p consecutive A-nice integers such that q =/= p 
and B = AU {p~~T}. Then 

(3.5) Cb = CAtqp 

Proof. Let r > p be the next A-nice integer. After cancehng cj for each of 
the common cychc components J of A and B, we may assume that A = [g, p — 2] U 
[p, r — 2] and B — [q, r — 2\ . Then (|3.5p is equivalent to 

Sp-l{Sp-2 ■ - -Sq) = {Sp-2 ■ ■ ■ Sq)tqp 

which holds by □ 

3.4. Commutation of weak strips and strong covers 

An initial pair {W, S) consists of a weak strip W and strong strip S with 
inside(VF) = inside(5'). A special case is an initial pair {W, C) where C is a marked 
strong cover. A final pair {W , S') consists of a weak strip W' and strong strip 5" 
such that outside(M^') — outside(S"). Since a marked strong cover C is a special 
case of a strong strip we can refer to a final pair {W , C). 



W 

w 



initial pair 



final pair 



Let (W, C) = {w V, w --^ u) be an initial pair. By associativity in Sn one 
always has a commutative diagram 



where x — v tij — cau, in which the arrows labeled by A represent left multiplication 
by OA and those labeled hy i,j represent right multiplication by . Commutativity 
of such a diagram means that the corresponding products of elements in Sn result 
in the same element. 
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We say that the mitial pair {W, C) commutes (or that W and C commute, or 
that ca and C commute, and so on) if u < that is, v{i) < v{j) or equivalently 
x{i) > x{j). This should not be confused with the commutation of a diagram. 

Let (W',C') = {u ^ X, V ^ x) he a final pair. We say that {W',C') 
commutes ii w < u (that is, w{a) < w{b) or equivalently u{a) > u{b)) where 
w = utab = c^}v. 

Lemma 3.14. Let w,u,v,x € Sn, A C 'L/n'L and tij be such that u = wtij, 
V = Caw, and x = vtij = cau. Then the following are equivalent: 

(1) {w V, w u) is a commuting initial pair. 

(2) {u X, V x) is a commuting final pair. 

Proof. For the forward direction we have 

\A\ + e{u) > e{x) > e{v) + i^\a\ + e{w) + 1 = |A| + e{u) 

A . 

since cau — x, v < x, w --^ v is & weak strip, and w < u. All inequalities must 

A . 

be equalities, which proves that u ^ x is a weak strip and v < x. This proves the 
forward direction. The reverse direction is similar. □ 

We list some Lemmata regarding noncommuting initial pairs. 

Lemma 3.15. Let {W, C) — {w v, w •) be an initial pair and c = ca- 

(1) The following are equivalent: (i) (W, C) does not commute; (ii) c{w{i)) > 
c{w{j)); (Hi) w{j) < c{w{i)), w(i) G A and w(i) is A-nice; (iv) w{i) is 
A-nice and the next A-nice integer after w{i) is larger than w{j). 

(2) // (W, C) does not commute then < 'w{j) — w{i) < n and w{j) is not 
A-nice. 

Proof. By definition, {W, C) commutes if and only if v{i) < v(j). The lemma 
then follows from Lemma [531 □ 



Lemma 3.16. Let {w v, w u) be a noncommuting initial pair and 



A — {w{j) — 1}. Then u v is a weak strip. 

Proof. By a length calculation it suffices to show that c^vu = v. By Lemma 
I3.15( iv). w{i) and w{j) are consecutive A^-nice integers. Applying Lemma 13.131 
with the consecutive A^-nice integers w{i) < w{j), we get Cy^v — CAtw{i)w{j) — 
CAtu{i)u(j) which easily gives the result. □ 

The picture for Lemma 13.161 is given below. 



w 9- u 



Lemma 3.17. Let u ^ v be a weak strip with \A^\ < ji — 1, q < p consecutive 



A'^ -nice integers such that u ^{q) < u ^(p), A' ~ A^ U {p — 1}, and x — ca'U. 
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Then 

(3.6) a; = CA' w = ■y t„-i(g)^„-i(p), 

A' 

u X is a weak strip, andv<x is a strong cover. If in addition u^^{q) < I < u^^{p) 
then {u^ X, v x) is a noncommuting final pair where (a, 6) = {u^^{q),u^^{p)). 

Proof. Let r be the next A^-nice integer after p. Using the hypothesis 
u'^{q) < u^^{p) and applying Lemma fS.lOl to the pairs of consecutive A^-nice 
integers q < p and p < r, and to the pair of consecutive A'-nice integers q < r, we 
conclude that u ca'U = a; is a weak strip. Equation (I3.6|l follows from Lemma 
13.131 The strong cover assertion follows from l\3.G\i and a length computation. 
Noncommutativity holds since q < p. □ 

Lemma 3.18. Let v he a weak strip with \ < n — 1, q < p consecutive 
-bad integers such that v^^{q) < v^^{p), A' = U {q}, and x — ca'U. Then 

(3.7) X = c^' u = t;i„-i(,)^^-i(p), 

u ^ X is a weak strip, and v <x is a strong cover. Moreover if v^^{q) < I < v^^{p) 
then (u ^ X, V x) is a noncommuting final pair where (a, b) — {v^^{q), v^^{p)). 

Proof. By Lemma and p.3p . qi — c^i (q) and pi = c^i {p) are consecutive 
A^-nice integers with pi — 1 = q. We have v^^{q) = it~^(c^J(g)) = u^^{qi) and 
'^'^^ip) — '^^^(Pi)- The Lemma follows by an application of Lemma 13.171 to the 
weak strip u v and the A^-nice integers qi < pi. □ 

The diagram for Lemmata 13 . 1 71 and 13 . 1 81 is given as follows. 



u 




We now give the corresponding lemmata for final pairs. 

Lemma 3.19. Let {W , C) = {u ^ x, v x) be a final pair. 

(1) (W' , C) does not commute if and only if u{a) < u(b). 

(2) If (W' , C) does not commute then u{a) g A', u{a) is A' -nice, u{h) is not 
A' -nice, CA'u{a) > u(b), and < u{b) — u{a) < n. In particular, the next 
A-nice integer after u{a) is in that case larger than u{b). 

Proof. The proof is similar to that of Lemma 13.151 □ 
Lemma 3.20. Let {u ^ x, v x) be a noncommuting final pair and A'^ = 



A' — {u{b) — 1}. Then u ^ v is a weak strip. 

Proof. By a length calculation it suffices to show that ca^u = v. By Lemma 
13.191 u{a) < u{b) are consecutive A^-nice integers. The result then follows by an 
application of Lemma 15.13! with the consecutive A^-nice integers u{a) < u{b). □ 
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The picture for Lemma 13.201 is given below. 

u 

A' 



a.b 



■ X 



A^ 

Lemma 3.21. Letu v be a weak strip with \A^\ < n— 1 andp < q consecutive 
-nice integers such thatu^^[q) < u^^[p), A = A^U{q — 1}, andw — c^^v. Then 

w = c']^^v = ui„-i(g),„-i(p), 



w V is a weak strip, and w — ^ u is a strong strip where {i,j) — [u ^(jq), u ^{p))- 
If also u^^{q) < I < u^^{p) then {w v, w u) is a noncommuting initial pair. 



Proof. The proof of the weak and strong strip properties is similar to the 
proof of Lemma 13.171 The noncommuting property is equivalent to ca (p) > ca (q) , 
which holds by the assumptions on p and q. □ 

The diagram for Lemma 13.211 is given below. 



2J 

W 5^ li 

I 

A I 

y . 

V 




CHAPTER 4 



AfRne Insertion and AfRne Pieri 



We employ Fomin's general method of growth diagrams l4l to help define a bi- 
jection that we call affine insertion. We deduce our main results from the properties 
of affine insertion. 



4.1. The local rule 0„ „ 

For M, u e Sn, let 1° ^ be the set of triples {W, 5, e) where {W, S) is an initial pair 
with outside(VF) = v and outside(S') = u, and e e Z>o is such that size{W) + e < n. 
Let C° „ be the set of final pairs {W , S') such that inside(iy) ~ u and inside(S") = 

V. 

Proposition 4.1. For each u,v e Sn there is a bijection 

(j)u,v '■ 1u,v * ^u,v 

iW,S,e)^{W',S') 

such that 

(4.1) size(S") = size(S') + e. 

The weak and strong strips form the edges of a diagram 




We call (j>u.v the local rule because of its role in the affine insertion bijection defined 
in the following section. The formulation of the local rule and the proof of its 
bijectivity form the core of our arguments and occupy Chapters [5] through [71 

4.2. The afRne insertion bijection <i>„ „ 

A n-bounded matrixis a matrix m = {mij)i_jyQ with nonnegative integer entries, 
only finitely many of which arc nonzero, all of whose row sums are strictly less than 
n. We write rowsums(TO) and colsums(m) to indicate the sequences of integers 
given by the row sums and column sums of m respectively. Let M denote the set 
of n-bounded matrices. 
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Fix u,v £ Sn- Let Xu,v be the set of triples (T, U, m) where T is a skew strong 
tableau, U is a skew weak tableau, and m £ A4, such that 

inside(T) = inside(C/) 
outside(r) = u 
outside(J7) = v 
wt(/7) + rowsums(m) < (n — 1, n — 1, . . . ) 

When u = V = id E S„ then Xid.id — since T and U must respectively be the 
empty strong strip and weak strip from id to id. 

Let Ou^v be the set of pairs {P, Q) where P is a skew strong tableau and Q a skew 
weak tableau with inside(P) = v, insidc(Q) = u, and outside(P) — outside((5). 

The following is our main theorem. The reduction of its proof to that of Propo- 
sition 14.11 is an instance of Fomin's theory [4j . 



Theorem 4.2. There is a bijection 
4.2) 

(r,C/,m)^(P,Q) 

such that 

(4.3) wt(T) + colsums(m) = wt(P) 

(4.4) wt(C/) + rowsums(m) = wt(Q). 



We picture the input and output of the bijection by the diagram 




where solid and dotted arrows indicate input and output data respectively. 



Proof of Theorem 14.21 For each u',v' e S^, Proposition 14.11 specifies a 
bijection <j)u',v' '■ ^u' v' ~^ ^u' v" {T,U,m) G Tu.v The growth diagram 

of {T,U,m) (defined by the local rules 4>u' ,v') is by definition the directed graph 
with vertices Gij G Sn for i,j € Z>o indexed matrix-style, with horizontal edges 
Gi,j-i Gij endowed with the structure of strong strips, and vertical edges 
Gi-i.j — + Gij given by weak strips, such that 

(Gl) The zero-th row of G is the strong tableau T. 
(G2) The zero-th column of G is the weak tableau U . 
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(G3) For every (j, j) G Z?,q, the two-by-two subgraph 



Gi-i, 



w' 



s' 



satisfies 



where u' — Gi^ij and v' = Gi.j_i. 

The north and west boundaries of the growth diagram are specified by (Gl) and 
(G2) above. Here by convention a strong (resp. weak) tableau is defined by an 
infinite sequence of strong (resp. weak) strips, but only finitely many of these 
strips have positive size. 

If (i, j) € Z?.o, by induction we may assume that the part of the growth diagram 
northwest of Gij-, is uniquely defined. Then by (G3), Gij and its incoming edges 
are specified by 4>u'.v'- It follows that the growth diagram is well-defined. 

The growth diagram satisfies two additional properties. 

(1) Let Gi,, denote the strong tableau given by the i-th row of G. Then for 
i ^ 0, Gi^t stabilizes; call the limiting tableau Goo,». 

(2) Let G,,j denote the weak tableau given by the j-th column of G. Then 
for j 3> 0, G,,j stabilizes; call the limiting tableau G, oo- 

To see this, there is an 3> such that U stabilizes after N steps and rriij = for 



i > N. By ()4.1|) every column G,.j stabilizes after N steps. In other words, for all 
i > N, the i-th row Gi^, is the same strong tableau. This proves the existence of 
Goo,.. In a similar manner one may show that G,.oo exists. 
The affine insertion bijection is defined by 

(4.5) $„,„(r,{/,m) = (P,Q) (Goo,., G.,oo). 

By construction (P, Q) £ Ou,v and ^u,v is well-defined. 

To show that is a bijection we define the inverse map ^u,v Given (P, Q) G 
Ou,v, let X = outside(P) = outside(Q). Let Ni and N2 be sufficiently large such 
that for all i > Ni and j > N2, the «-th element of P and the j-th element of Q 
are equal to x. We define a growth diagram G as follows. We set G^,. = P for 
i > Ni and G,j = Q for j > N2- This is consistent: these definitions overlap in the 
region i > Ni and j > N2, where the entries Gij are all equal to x and all strong 
and weak strips are empty. In the middle of each two-by-two subdiagram in this 
region we place the integer 0. We now use the inverse ipu',v' of the local rule 4'u',v' 
to fill in each two-by-two subdiagram of G (including the "excitation integers" in 
the middle of the subdiagram) given its south and east borders. Then all of G may 
be computed, as well as a matrix m. Letting T — Gq,. and U — G.,o be the north 
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and west boundaries of G, we define ^'(P, Q) = {T,U,m). ft is easy to show that 
"^u,v o ^u,v = idx„,„ and o „ = ido„^. □ 

As a special case of affine insertion (and using Proposition 12. 6p we obtain an 
RSK bijection for the affine Grassmannian. 

Theorem 4.3. For I — 0, 'f'id.id gives a bijection A4 — > Oid.id from n-hounded 
matrices to pairs (P, Q) of strong and weak tableaux from id to a common 0- 
Grassmannian element outside(P) — outside(Q). 

We shall prove later (Theorem If 0.4p that when n oo the affine insertion 
bijection 'f'id.id coincides with the classical row insertion RSK correspondence. 

4.2.1. Cauchy identity and Pieri rules for strong and weak tableaux. 

Define the affine Cauchy kernel VLn{x,y) by 

^n{x, y) = n (1 + V^hi{x) + y1h2(x) + ■■■ + y1~^K-i{x)) 

i 

= X! hx{x)mx{y). 

A : Ai <n 

It is an element of a completion A(„)(x)(g)A(")(y) of A(„)(x) (g) A(")(j/). 

The following is an immediate enumerative consequence of Theorem 14.21 

Theorem 4.4 (Generalized AfRne Cauchy Identity). Let u,v e 5„. Then the 
following identity holds in the quotient 1j[[xi, X2, . . . ,yi,y2, ■ ■ •]]/ (y" , 2/2 j ■ ■ ■) '^f 
formal power series ring in two infinite sets of variables: 

^7i{x,y) Strong„/„(a;)Weak^/„(y) ^ Strong^/„(x)Weak^/„(y). 

Corollary 4.5. Let w, w e 5,*]. Then the following identity holds in the quo- 
tient of the formal power series ring X2, ■ ■ ■ , yi, y2, ■ • ■]]/ (y" , y2 i ■•■)■' 

^nix,y) Strong„/„(a;)Weak„/„(y) ^ Strong^/„(a;)Weak^/„(y). 

w£S» zes° 
Proof. In Theorem 14. 4[ use Proposition 12.61 and Lemma [3. 121 to deduce that 
Strong2/„(a;) = unless 2; G S*^ and Weak„/„,(y) — unless w £ S^^. □ 

Corollary 4.6 (Affine Cauchy Identity). The following identity holds in the 
formal power series ring Z,[[xi, 2:2, ■ • ■ , yi, y2, • ■ ■]] •' 

Qnix,y)= Y Strong^ (a;)Weak^(?/). 

Proof. Put u = w = id in Corollary 14.51 D 

We now deduce two combinatorial Pieri rules from Theorem 14.21 Recall from 
Example 12.91 that hr{x) = Strong^^ ^(2;)- 

Theorem 4.7 (Strong Pieri rule). Let u ^ Sn and 1 < r < n - I. Then 

hr{x) Strong^{x) = ^ Strong^ (x), 
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where the summation is over weak strips u z of size r. 

Note that by Proposition 12. 6[ Strong„(a;) = unless u is Z-Grassmannian so all 
the permutations in the theorem can be taken to be ^-Grassmannian. 

Proof. In Theorem l4.21 set v = id and restrict the bijection to triples {T, U, m) 
such that m has non-zero entries only in the first row and such that the entries in 
the first row of m sum to r, where 1 < r < n — 1. Since v — id, we must have 
inside(T) = inside(t/) = id as well, so that in effect $„,t, restricts to a bijection 
from pairs (T, m') to pairs (P, Q) where T is a strong tableau of shape u, the infinite 
vector m' given by m^- — mij has non-negative integer entries summing to r, and Q 
is a weak strip of size r with inside((3) = u, and finally P is a strong tableau with 
shape z = outside(P) = outside((5). Now note that the weight generating function 
of the vectors m' is hr{x). In view of (|4.3p . taking the strong tableau generating 
functions for the input and output of the bijection gives our theorem. □ 

One may define dual weak strips w-^v by replacing cyclically decreasing per- 
mutations with cyclically increasing permutations (defined in the obvious way). 

Theorem 4.8 (Dual strong Pieri rule). Let u G Sn and 1 < r < n - 1. Then 
er(a;)Strong„(x) = ^ Strong^ (a;), 

where the sum runs over dual weak strips u-^z of size r. 

Proof. In Theorem 14 . 1 1 1 we show that Strong„(a;) is a fc-Schur function when 
u G Sl^ and by ProDOsition l2.61 Strong^ (x) = otherwise. It is known from (21, that 
fc-Schur functions behave well under the involution to of A(^n) s-nd in our notation we 
have CL;(Strong„(a;)) = Strong^j* (x) where u* is defined in (j3.2p . The involution u ^ 
u* also interchanges cyclically increasing and cyclically decreasing permutations. 
Thus the dual strong Pieri rule follows from applying uj to Theorem 14.71 □ 

Recall from Example 13.71 that in A^"^ hr{x) — Weakc, ^{x). 

Theorem 4.9 (Weak Pieri rule). Let w £ Sn and 1 < r. Then the following 
identity holds in A^"^ ; 

hr{x)Weak^{x) = ^ Weakoi,tsidc(s)(a;), 
s 

where the sum runs over strong strips S of size r such that inside(S') = w. 

Proof. The proof is similar to the proof of Theorem 14. 71 We take u = id, and 
restrict to matrices m with non-zero entries only in the first column. □ 

For the weak Pieri rule, we can use Theorem 13.41 to give a dual rule. 

Theorem 4.10 (Dual weak Pieri rule). Let w G S„ and 1 < r. Then the 
following identity holds in A'"-* ; 

er(a;) Weak,„(x) = ^ Weakoutsido(S)-i (a^), 
s 
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where the sum runs over strong strips S of size r such that inside(S') ~ w ^ . 

One can replace w"^ and outside(5)~^ above by w* and outside(S')*, where w* 
is defined in ()3.2|) . 

Proof. Clearly uj^{hr) = as elements of A*^"). The theorem follows from 
applying aj+ to Theorem 14.91 and using Theorem 13.41 □ 

4.3. Pieri rules for the affine Grassmannian 

For this section, we will assume that / — and write "Grassmannian" instead 
of 0- Grassmannian. We will use Theorem 1 1.1 1 to deduce two Pieri rules for the afhne 
Grassmannian Gr. 

Theorem 4.11 (Monomial expansion of a fc-Schur). Let u G 5° be Grassman- 
nian. Then Strong„(2;) coincides with the k-Schur function s^\x) for k = n — 1. 

We conjecture in Conjecture 19.111 that the fc-Schur functions depending on a 
parameter t can also be expressed as generating functions of strong tableaux, using 
an additional statistic called spin. 

Proof. The fc-Schur functions s^\x) are defined in [21] as the symmetric 
functions satisfying a certain Pieri rule. Equivalently, one may define {si'^^' (x) \ 
u G S^} as the basis of A(„) dual to the affine Schur basis {Fu{x) \ u G S^} of 
A(") see [221 IS]- By definition we have Weak„(a;) = F„(a;). By Corollary SH it 
suffices to show that Strongj„(a;) G h.{n), for then the duality (and the fact that 
{Strong^(x) I w G S*"} forms a basis of A^n)) will follow from an argument similar 
to [291 (4.6)]. 

To show that Strong^ (x) G A we let Ui be the ring- involution of the ring 
of formal power series in a;i,X2,..., which interchanges Xi and Xi+i. Then by 
Corollary in 

^ {a, ■ Strong„(x)) Weak^(2/) 

= CT,; ■ ]J (1 + y^hiix) + y1h2{x) + ■■■ + y^-^hn-iix)) 

i 

= n (1 + y^'T'ii^) + yf'^2{x) + ■■■ + yr^hr,-i{x)) 

i 

= ^ Strong^ (a;)Weaku,(y). 

By Theorem 13.51 the Weaku;(x) are linearly independent elements of A*^"^. Taking 
the coefficient of Weak^, (y) in the above equation we obtain 

Strong,„(x) = CTj • Strong„(x). 

Since this holds for all z, we have Strong^(a;) G A. Finally, {Weak^(a;) | w G 5°} is 
independent and no terms hr{x) for r > n — 1 occurs in the affine Cauchy kernel; 
so Strong„(a;) G A(„). □ 
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Thus by Theorem 1 1.!) the generating functions Strong^ (x) are exphcit combi- 
natorial representatives of ^u, . 

Recall from Examples 12.91 and 13 . 71 that ■= Si+m-i • ■ ■ Si+iSj. 

Theorem 4.12 (Fieri rule for _ff»(Gr)). Let w e S° and 1 < m < n - 1. Then 

where the sum runs over weak strips w z of size m. 

Proof. Example 12.91 savs that Strong^^ ^ (x) = hm{x). The theorem then 
follows immediately from Theorems 14.71 14.111 and 11.11 □ 



The Fieri rule for fc-Schur functions was first stated in |21j and in the notation 
here in [23j . Combining this with the geometric identification in 1161, one can 
obtain Theorem 14.121 directly as a corollary. 

Theorem 4.13 (Fieri rule for H*{Gv)). Let w <^Sl andl< m. Then 

S 

where the sum runs over strong strips S of size ni such that insidc(S') — w. 

Froof. Example 13.71 savs that Weakens (a;) = hm{x) in A^"). The theorem 
then follows immediately from Theorems 14.91 and 11.11 □ 

It is not difficult to see that both the weak and strong Fieri rules reduce to 
the classical Fieri rule for the finite Grassmannian when i{w) + m < n. Using 
the involution w i-^ w* oi p.2p . Theorems 14.81 and 14.101 also gives us a rule for 
muhiphcation by ^^o-" in 7J,(Gr) and H*{Gr) (note that if w e 5"° then w* S S°). 

Proposition 4.14. Theorems \4-. 12\ and \4-. 1 S\ determine the multiplicative struc- 
tures o/H,(Gr) andH*{GT). 

Froof. It suffices to show that the Fieri rules can be inverted, so that (or 
^u,) for any w G 5° can be written in terms of {S,'^" "^} (or {^cq ,„})■ The theorem 
will then follow from the fact that j^"" | w e S*"} (or j^^, | w G 5"°}) forms a basis 
of ff*(Gr) (or H,{Gr)). 

The transition matrix between {Weak^(a;) | w G 5"°} and {h\{x)}, given by 
Theorem 14.91 is the same as the transition matrix between the monomial symmetric 
functions {m\{x)} and the strong Schur functions {Strong^(a;) | w G S^}. Note 
that {h\{x)} does not form a basis for A'"', so the matrix is "rectangular" (and 
infinite). Since {Strong^ (x) | w G S^} is linearly independent, the matrix has 
full rank (when restricted to submatrices of each degree) and so the Fieri rule 
can be inverted to write Weakuj(a;) in terms of h\{x). Applying Theorem 11.11 we 
may write ^"^ in terms of ^'=0.'". Similarly, the fact that {Weaku,(a;) | w G 5°} 
is linearly independent allows one to write {Strong.|^(a:) | w G 5°} in terms of 
{hx I Ai < n}. □ 
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4.4. Conjectured Pieri rule for the affine flag variety 

Theorems 14.91 and 14.131 suggest that we make the foUowing conjecture. In the 
following conjecture, we let / be arbitrary again. 

Conjecture 4.15 (Conjectured Pieri rule for H*{Q/B)). Let w g Sn and 
1 < TO. Then in H*[Q/B) we have 

f-ci,m /-w ^outside(S) 

t.B SB — t.B ' 
5 

where the sum runs over strong strips S of size m sueh that inside(S') — w. 

In jl6l it is observed that Weak^(a;) — Fw{x) is the puUback of from 
H*{g/B) to H*{Gt) under the map Gr ^ G/B induced by the map nSU{n) 
LSU{n) — > LSU{n)/T where flSU{n) denotes the based loop-space, LSU{n) de- 
notes the loop space, and T denotes the maximal torus. The puUback of Conjec- 
ture [415] is consistent with Theorem 14.91 

Remark 4.16. Recurrences for the structure constants of H*{Q/B) are given 
in |13j . It may be possible to derive Coniecture l4.15l directlv from these recurrences. 

Remark 4.17. Coniecture l4.15l is consistent with the Pieri rule for the classical 
finite-dimensional flag manifold. Indeed ii w,z G Sn C Sn and I G [l,n], then the 
existence of a marked strong cover w z is exactly the combinatorial condition 
appearing in Monk's rule, while strong strips agree with the "path formulation" of 
the Pieri rule in [37\. Note that in |37j . using the language that we have introduced, 
a strong cover w z would be "marked" at w{i) rather than w{j). However the 
Pieri rules obtained from the two different markings agree. 

4.5. Geometric interpretation of strong Schur functions 

In this section we list some conjectural properties of strong Schur functions, 
assuming I = for simplicity. 

Conjecture 4.18. Let u,v S £"„ he two affine permutations. We have the 
following successively stronger properties. 

(1) We have Strong^y^, (x) G A. 

(2) FFe /lawe Strong„/^(a;) G A(„). 

(3) We have Strongj^/j,(a;) = X^^iesf "^"to Strong^ (x) where is defined by 

The corresponding properties of weak Schur functions are known. Symmetry 
(Theorem 13. 3|) was proven combinatorially in |15j (see also |22j ) while positivity 
was shown in [16j using geometric work of Peterson |32j . 

Proposition 4.19. Conjecture \4.18\ follows from Conjecture \J^.15\ 

Proof. By making the identification A^"^ = i/*(Gr), wc may consider the 
affine Cauchy kernel ri„ as an element of the completion A(„)®i/*(Gr). If Conjec- 
ture [3112] holds, then it completely determines the action of H*{Gy) on H*{Q/B), 
obtained from the inclusion H*{Gi) d H*[Q/B). 



4.5. GEOMETRIC INTERPRETATION OF STRONG SCHUR FUNCTIONS 



27 



Let (. , denote the inner product on H*[Q/B) defined by {^"^ , S,b) — ^wv 

By the definition of fin and Conjecture 14. 151 we have 

Strong„/^(a;) = (17„ • Cb , 

where fin ■ G A(„)(g)i/*(^/S). By CoroUarv 14.61 and Theorem ll.il we may also 
write 

strong^ (a;)®r 

so that 

Strong„/„(x) = ^ StTong^XmBCB,CB)g/B- 

wes° 

Thus Strongly.;, (x) £ A(„) . By general positivity results of [H I14j , we have c^^ S 
Z>o. □ 



CHAPTER 5 



The Local Rule 



In this section we shall define a local rule as in Proposition 14. li as a sequence 
of operations called internal and external insertion steps. 



5.1. Internal insertion at a marked strong cover 

Let C — {w u) be a marked strong cover. Internal insertion at C is a 
map that takes as input, a final pair of the form (W, S'l) — (w v, ■ — ^ v) and 
produces an output final paii0 of the form {W , S') ~ (u x, ■ -^-^ x) such that 
inside(VF') — u = outside(C), size(W^') = size(VF), inside(S") — inside(S'i), and 
size(S") = size (5*0 + 1. This given, we define 

C = last(S") = {v' ^ x). 

Internal insertion has three cases named A, B, and C. In Cases A and B the output 
takes a particularly pleasant form: S' is obtained by appending C" to S[ and v' — v. 
In Case C, S' is obtained from S'l by placing a strong cover just before the last cover 
of S[. 

If size(S'() > we write 

(5.1) S[ = { ^v). 

We need only specify A' , (a, 5), and the rule for obtaining S' for then we set 

X — CA'{u) ~ outside(S'') 

v' = Xta,b- 

5.1.1. Commuting case. 

Case A (Commuting case) Suppose {W, C) commutes 
(i,j), and 5" = S{UC'. 

Example 5.1. Let n = 4, / = and C = ([3,5,-2,4] 
sider internal insertion at C of the final pair 

[W, S[) = ([3, 5, -2, 4] ^4 [4, 5, -2, 3], [4, 5, -2, 3] - 



Set A' = A, (a, b) = 
[1,7,-2,4]). Con- 
. [4,5,-2,3]). 



^This is an abuse of language: in tlie generality in which we define internal insertion, S" is only 
guaranteed to be a strong tuple, not necessarily a strong strip. However, whenever we apply 
internal insertion in the definition of the local rule 4'u,v, S' will be a strong strip. 
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Since (W, C) commute, the output final pair is 

{W\ S') = ([1, 7, -2, 4] [1,8, -2, 3], [4, 5, -2, 3] ^ [1, 8, -2, 3]). 
where S' = ([4, 5, -2, 3] [1, 8, -2, 3]). 

5.1.2. Noncommuting cases. Otherwise, we assume that (W, C) does not 
commute. Since w{j) — u{i) is not A-nice by Lemma |3.15[ we can let 

(5.2) po = u{i) - 1 

(5.3) A''=A-p^. 

Case B (Normal bumping case) Suppose that (W, C) does not commute and 
either size{S[) — 0, or size{S[) > and i ^ ai. Let q < p he the unique pair of 
consecutive A^-nice integers such that q < u{j) and u^^{q) < I and q is maximal. 
We set A' = U {p^}, (a, b) = (^-^(g), ^-^(p)), and S' ^ S[U C . 

Example 5.2. Let n — 6,1 — and 

C = ([5, 0, 1, 9, -2, 8] ^ [3, 0, 1, 11, -2, 8]). 

Consider the final pair {W, S[) given by 

W = ([5, 0, 1, 9, -2, 8] ^^^^^ [4, -1, 1, 12, -3, 8]) 

and S[ = ([4,-1,1,12,-3,8] — > [4,-1,1,12,-3,8]). The pair (W,C) does not 
commute and we have po — 4 and = {3, 5}. Since size(5() ~ 0, we are in Case 
B. One calculates that q — 2,p — 3, so that A' — {2, 3, 5} and (a, b) — (0, 1). Thus 

the output final pair is given by W = ([3, 0, 1, 11, -2, 8] ''^^^^ [2, -1, 1, 12, -3, 10]) 
and S' = ([4, -1, 1, 12, -3, 8] [2, -1, 1, 12, -3, 10]). 

Case C (Replacement Bump) Suppose that {W, C) does not commute, that 
size{S'i) > 0, and i = ai. Let q < p he the unique pair of consecutive A^-had 
integers such that q < po and y^^{q) < I and q is maximal. Set A' = U {q}, 
{a~ ,b~) = {y^^{q),y^^{p)) and let S' he obtained by inserting (a~,5~) just before 
the last pair of indices (ai, 6i) = (i, bi) of S'l- 

Example 5.3. Let 4,1^0 and C ^ {w ^ [1, 7, -2, 4] ^ u = [1, 8, -2, 3]). 
Consider internal insertion at C of the final pair 

(W, S[) = ([1, 7, -2, 4] [1, 8, -2, 3], [4, 5, -2, 3] ^ [1, 8, -2, 3]), 

where S[ = (y = [4,5, -2,3] = [1,8, -2,3]). Since the pair (W,C) does not 

commute, size(5() = 1 > and i = oi = — 2 we are in Case C. We have po = 3 and 
= so all integers are A^-bad. We find that q = l,p = 2, A' = {1} and obtain 

5' - ([4, 5, -2, 3] ^ [4, 6, -3, 3] ^ [2, 8, -3, 3]). 

Note that W = ([1, 8, -2, 3] [2, 8, -3, 3]) is indeed a weak strip. 

5.1.3. External Insertion. 
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Case X (External Insertion) Let W — {w ^ v) he a. weak strip sucti tiiat 
1^1 < n — 1. Let q < p he the unique pair of consecutive A-bad integers such that 
v~^{q) < I and q is maximal. We set A' ~ AU {q} and (a, 6) {v~^ {q),v~^ (p)). 

By Lemma 13.181 we have a noncommuting final pair 

External insertion is the map that takes as input a final pair (W, S'l) = 

V, ■ — > v) with size(W^) < rt — 1, an 
W ^ {W, C) as above and S' ^ S[U C . 



A S[ 

{w V, ■ — > v) with size(W^) < rt — 1, and produces the final pair {W , S'), where 



Example 5.4. Let n = 5, / = and = ([2,-4,5,8,4] ^iw^ [2,-5,6,9,3]). 
We find q — 4 and p — 6. Then external insertion of {W,S'i), where S[ — 
([2, -5, 6, 9, 3] — > [2, -5, 6, 9, 3]) produces the final pair {W, S') where 

([2,-4,5,8,4] [2,-5,4,11,3]), 

and S' = ([2,-5,6,9,3] [2,-5,4,11,3]). 

5.2. Definition of 0„ „ 

Fix u,v £ Sn- We define the value of (j)u,v on {W, S, e) G 1°^^ as the result of a 
sequence of steps. Each step, which is either an internal or external insertion, takes 
a final pair and produces another. 

We start with the final pair 

(^(0)^ 5(0)) = (^w,v — > v) where S'-°^ is the empty 
strong strip from v = outside(VF) to itself. Iteratively, for 1 < k < m = size(S'), 
perform the internal insertion on the final pair {W'^''^^\ S^''^^'^) at Ck, and let 
(Vl/(fe)^5'(fe)) be the resulting final pair. The result of this sequence of internal 
insertions is the final pair (W^("),S'(™)). We now perform e external insertions. 
For m < k < m + e define 

We define 4>u,v{W,S,e) = (W',S') (vt/(™+e), fi-C'^+e)) to be the final pair pro- 
duced by this process. 

5.3. Proofs for the local rule 

We now establish the well-definedness of the local rule <j)u.v and some of its 
properties. 

5.3.1. Case X. By construction we have a commutative diagram 

(5.4) w u 

I 

W W \ A' 

C 1 
V ^ a; 

a,b 

where (a, 6) = {v^'^{q),v^^{p)). By Lemma [5.51 applied to S[, S' is a strong strip, 
finishing this case. 

Lemma 5.5. In Case X let W t-^ {W',C'). Then for any strong strip S such 
that outside(5') = v , S' = S U C is a strong strip. 
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Proof. If size(S') = then 5' is automatically a strong strip. Otherwise let 



last(S') — (• v). By the maximality of g, we have v(i) < q since otherwise there 
would exist a pair of consecutive ^-bad integers r and s such that r < v{i) < s with 
v~^{s) > I > i, which would contradict Lemma 13.101 for the weak strip {w ^ v). 
Therefore m(C') — p > q > v{i) = m(last(S')), so S' is a strong strip. □ 



By Lemma [5.51 each external insertion sends a final pair to a final pair, preserves 
the inside permutations of both the weak and strong strip, and adds one to the sizes 
of the weak and strong strips. Thus to check that (f)u,v is well-defined, we may reduce 
to the case e = where no external insertions are required. 

5.3.2. For internal insertion cases. We want to compute 4'u,v{W, S, 0) with 
size(S') = m. By induction we may assume that all of the internal insertions 
have been performed except the last, which computes the internal insertion on 
(M/(™-i),S'("-i)) at Cm, resuhing in S-^")). To avoid the proliferation of 

subscripts and superscripts we change notation, forgetting the global meaning of 
u,v,W,S. We denote this last internal insertion step as the internal insertion on 
(W, S[) at C, resulting in {W , S'). We write 

{W, C) = {w V, w — i- u) 
S : w~ ^ w ^ > u 



(5.5) S[: ■■■"-Ifl^y 



V 



a .h a ,h a,h 

S' : y ^ v' 7^ X 

c'-- c'- C 

{W',C') = {u^x, v' ^x) 
We use the following induction hypothesis. 
Property 5.6. 

(i) (a) m(C") < m(C) in Case B and (b) m(C") = ca'(to(C))) in Cases A 

and C. 
(u) x{b) < m{C). 

(iii) Case C cannot be preceded by Case B, and if Case C holds then i~ = i. 

(iv) The final pair {W , C) commutes in Cases A and C and does not commute 
in Case B. 

5.3.3. Case A. 
(5.6) • ■ • 5~ w ^ u 



A 



A 

V 



V ^ X 



By Lemma 13.141 {u x, v x) is a commuting final pair. 
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Lemma 5.7. In Case A, Property \5.6[ is satisfied. 
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Proof. We have m{C') — x{i) — CAu{i) = c^(r7i(C)), which proves (i). (ii) 
is equivalent to CAw{i) < w^), but this foUows by Lemma 13.151 (iv) was proved 
above. □ 

Lemma 5.8. In Case A, S' is a strong strip. 



Proof. We use the notation in (|5.6p where the top row gives S and the bottom 
row gives S'. By induction 5*^ is a strong strip, so we need only check that m{C'~) < 
m{C'), that is, v{ai) < v{j). 

Since S* is a strong strip w{i~) < w{j). By induction, Propertv 15.6( 1) asserts 
that either (a) v{ai) < w{i^) or (b) v{ai) — CA{w{i~)). Suppose (a) holds. By 
p.4p . v{j) = CA{'w{j))) > w{j) — 1 > w{i^) > v{ai), as desired. 

Suppose (b) holds. Then = ai, since v{i~) = CAw{i~) = v{ai). In particular 
w(ai) < wij). 

We shall assume that v{ai) > v{j) and derive a contradiction. If v{ai) = v{j) 
then we have the contradiction I > ai = j > I. So we assume v{ai) > v{j). In 
other words w{ai) < w{j) is inverted by ca- By Lemma 13.91 w{ai) is A-nice, 
CAw{ai) > w{j), and CAw{j) — w{j) — 1. By induction, Propertv 15.6( 11) gives 
v{bi) < w{i^) ~ w{ai) < w{j) — 1 = v{j). So 

(5.7) v(bi) < v{j) < v{ai). 
Next we have 

(5.8) ai < ; < 6i < j. 

This follows from I < j and j ^ (ai,6i), which follows from Lemma 12.21 with the 
strong cover C" = (• v) and (|5.7|) . We have 

(5.9) v{bi) < v{i) < v{j) < v{ai). 

This is obtained from (|5.7|) using Lemma 12.21 for the strong cover C = {v ^ •) 
and i < I < hi < j. We have 

(5.10) i < ai < I < hi < j 

since i (ai, 5i) by Lemma for the strong cover C . We have 

(5.11) w{ai) < w{i) < w{j). 

This follows from Lemma for the cover C, i < ai < I < j, and w{ai) < w(j). 

Now CA inverts w{ai) < w{j). By Lemma [3.91 and (|5.11[) . ca inverts w(ai) < 
'w{i) as well. With (|5.10p . Lemma [3. Ill gives a contradiction. □ 



5.3.4. Case B. We first sketch the proof of Case B and fill in the proofs 
afterwards. Since {W, C) does not commute, by Lemma l3.161 w --~> w is a weak strip 
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such that the diagram commutes: 

(5.12) w-^u 



Recall that in Case B, {a,b) — {u ^{q),u ^{p)), A' — U {p — 1}, and x — ca'U. 

Using Lemma rS.lOl we may apply Lemma [3.171 which says that (w ^ x, v x) 
is a noncommuting final pair such that the diagram commutes. 

W 5~ U 

/ I 

/ 1^' 

V ^ X 

a,b 

Lemma [5.121 shows that S' is a strong strip, completing the proof sketch for Case 
B. 

Lemma 5.9. In Case B, let 

(5.13) q' — u{i) — n, 

(5.14) p' = uU). 
Then 

(5.15) u-\q') <l <u-\p'), 

(5.16) 0<p'-q<n, 

q' is -nice, p' is -nice and A-nice, and \A^\ = \A\ — 1. 

Proof, q' is A^-nice by definition, p' is A-nicc and A^-nice by Lemma [3. 151 
We have p' — q' = u(j) ~ u{i) + n = w(i) — w{j) + n. The second inequality 
in (j5.16p holds since C is a strong cover. Lemma 13.151 says that w(i) < w(j) are 
inverted by ca, and in particular, w{j) = u{i) is not A-nice and w{j) — w(i) < n, 
which implies that \A^\ = |A| — 1 and the first inequality in (|5.16p . 

By straddhng, u^^(g') — i — n<i<l<j — u^^{p'), so (|5.15p holds. 

□ 

Lemma 5.10. In Case B, q is well-defined and 

(5.17) q' — u{i) — n < q < p < u{j) = p' 

(5.18) u~\q) <l <u-^{p) 
Moreover \A'\ = \A\, q is A' -nice and p is A-nice. 

Proof, q is well-defined, is A^-nice, and satisfies the above inequalities by 
Lemma 15.91 which assures that q is the maximum of a set that contains q'. p is 
A-nice and satisfies the above inequalities, also thanks to Lemma [531 q is A'-nice 
since it is A^-nice and too close to p and distinct from p to be congruent to p, by 
(15J71) and ^JE^j. Since p is A^-nice it follows that \A'\ = + 1 = □ 
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Lemma 5.11. In Case B Property 1 5. (A holds. 

Proof. Property I5.6r iv) was already proved above. Note that if m{C') < 
m{C) then Propertv 15.6^ 11) foUows immediately: we have x{b) < x{a) — m{C') < 
m{C) since C is a strong cover. 

To prove Propertv l5.6f i). we have 



u{i) — 1 if p = u{j) 
ca{p) otherwise. 



(5.19) m(C') = xiu-\q)) = CA-{q) = c^v (p) = 

The first two equalities hold by definition. The third follows from (I3.3|l . For the 
last equality, u{j) is A-nice by Lemma [3. 151 If p = w(j), then by Lemma [3. 151 u{i) 
is the next A^-nice integer after u{j), and c^v (p) = u{i) — 1 by (|3.3p . Otherwise 
by (|5.17p p < u{j) and the cyclic component of p is the same in A and . Either 
way the last equality holds. 

For p — u{j), (|5.19p implies that m{C') < u(i) — m{C). For p < u{j), 
since u{j) is A-nice, by (|5.19p and the fact that C is a strong cover, p.3p gives 
m{C') — ca{p) < u{j) < u{i) — m{C) as desired. □ 

Lemma 5.12. In Case B, S' is a strong strip. 

Proof. We use the notation in the diagram below, where the top row is S and 
the bottom row is S' . 




By induction it suffices to show that m{C' ) < m{C'). Since v = c^vw (see (|5.12p ) 
this is equivalent to 

(5.20) CAv(u(ai)) < CAv(p). 
Since p is A^-nice it suffices to show that 

(5.21) u{ai)<p. 
Since S is a strong strip we have 

(5.22) w{i-) < w{j). 

The cases of Property 15. 6^ 1 lead in (a) to w{i^) > CAw{ai) > w{ai) — 1 from (|3.3p . 
and in (b) to CAw{ai) — CAw{i~). We thus find that 

(5.23) w(ai)<w(r). 

Suppose that ^ {i,j}. Then u{ai) < w{i^). We claim that 

(5.24) u(ai) < uij). 

li i^ = i then since Case B has i ^ a\ this implies u{a\) < w{i) = u{j) and (|5.24p 
holds. So we assume i" ^ i. Suppose w{i~) > w{i). Since C is a strong cover 
and i^ < I, by Lemma [2721 and l|5.22p we have i^ < i < I < j. By Lemma [3.1 5 ( iv) 
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and (|3.3p . ca inverts w{i) < w{i^) since w{i) < w{i^) < w{j), thus contradicting 
Lemma [3. Ill Therefore w{i^) < w{i), which, together with (|5.23p yields (|5.24p . 

Let qi be the maximum A^-nice integer such that qi < u{ai). By Lemma |3.10[ 
u~^{qi) < ai < I. By the definition of q and (|5.24p . we have q > qi. But q < p and 
p is A^-nice so (j5.2ip holds. 

Next suppose lii — j . Let ai = j + kn for some fc G Z. Since ai < I < j we 
have fc < and we have u{ai) = u{j) + kn < u{j) — n < u{i) — n < p by (|5.17p . 
proving (|5.2ip . 

Finally suppose ~ i but ai ^ i. Let ai = i+kn for some k ^ 0. By (|5.22p and 
(|5.23p we have w{ai) < 'w{j), that is, w(i) + kn < w{j). Now < w{j) — w(i) < n 
by Lemma [3. 151 so fc < 0. We have 

u{ai) — u{i) + kn < u{i) — n < p 

by (|5Tfp . proving ([OT]) . □ 



5.3.5. Case C. We first sketch the overall strategy for Case C using diagrams 
and fill in the proofs afterwards. By Lemma [3.161 u v is a weak strip such that 
the diagram commutes. 



(5.25) 



w ^ u 



i,bi 

By Lemma 15.151 (u ^ w, y v) is a commuting final pair. Let w' — uti^b^ 



[ly. By Lemma [3.141 {w' 



(5.26) 



^ y, w' — i u) is a commuting initial pair. 

i,b-i 

w' - - 

/ 

/ 




Let (a , 6 ) = (y ^iq),y ^{p)) and v' = tq^y = y^a-.f,-. Using Lemma [5.161 we 
final pair such that the diagram commutes. 



may apply Lemma [3.181 which shows that [w' v' , y ^ "* u') is a noncommuting 




w' ^ u 

I 

A' I 
V 

y--^v' 

a ,b 

Let X — CA'U — v'ti,b-^. By Lemma 15.181 the initial pair {w' v' , w' u) 
commutes, so that by Lemma l3.14l we have a commuting final pair {u x, v' 
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(5.27) w' ^ u 




A' 



I A' 
V 

y ^v' - - ^x. 

This shows that the definition of S' produces a strong tuple. Lemma 15.201 shows 
that S' is a strong strip, completing the argument for Case C. 

Lemma 5.13. In Case C, Provertv 1 5. bY iii) holds. 

Proof. Case B could not have occurred in the previous step for otherwise we 
would have CAw{i) = v{i) = m{C" ) < m(C~) < miC) — w{j), contradicting 
Lemma 13.151 

So the previous step must be Case A or Case C. The input reflection for that 
step is In Case A the output reflection is also and in Case C it 

has the form ■). But the output reflection for this step is (i, bi) so — i. □ 

Lemma 5.14. In Case C, 

(i) If j ^ hi then po = y{i). 

(ii) Otherwise j — bi + kn with k > 0,po ^ y{i) + kn and 6i — i < n. 
Furthermore, letting 

(5.28) qo^y{bi)~n, 
we have 

(5.29) < po - qo < n 

(5.30) l<y-\po). 

Proof. Since last(S'J) is a marked strong cover, we have 

(5.31) y{i)<y{bi) and i<l<bi. 
To prove (|5.29p . we have 

(5.32) y{bi) = yti^b^{i) ^ v{i) = CAw{i) = CAu{j) 
so that 

(5.33) go = CAu{j) - n. 

Then po — Qo = (""(*) ~ 1) ~ (caw(j) — n) = n — 1 — {cAu{j) — u{i)) and (|5.29p is 
equivalent to < CAu{j) — u(i) < n — 1. But this follows from 

(5.34) CAu{j) - n < u{j) < u{{) < CAu{j), 

which holds by p.3p . the fact that C is a marked strong cover, and Lemma [3.151 
So (j05D holds. 
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We have 

Po = u(i) - 1 = CA{u{i)) = CA{w{j)) = v{j) 

(5.35) ^ ^ ^ \y{i) + kn if j = 6i + fcn for some /c G Z. 

lyO) otherwise. 

This gives that in case (i) po = which yields (|5.30p . 

Otherwise, we are in case (m), with j = bi + kn for some fc G Z and po — 
y{i) + kn. We thus need to show that fe > and &i — z < n. By (|5.34[) and 
(|5.32p we have y(bi) — n < u(i) — 1 = pq. Substituting for pq using (|5.35p we have 
y{bi)-~n < y{i) + kn, which gives that A; > since y{i) < y{bi). Now, suppose k — 
so that j = bi and po = y{i). By Property 15. Gf iii) and Lemma [5.131 Case A must 
have occurred in some step and Case C must have occurred in every step thereafter. 
Let (Ci, C2, . . . , C,- = C) be the corresponding sequence of marked strong covers 
in S (starting with the above instance of Case A) with Ck ~ {uk-i Uk), where 
the first index of the reflection is always i. The Case A step had the same output 
reflection as its input reflection In the output strong strip, the applications 

of Case C inserted reflections just before this reflection, which stayed at the end, 
so that — {hj)- Now, since Ci is a strong cover we have Ui{i) > Ui{j). And 

since each Ck for A; > 1 is a strong cover we have Uk{i) > Uk-i(i) > Uk-iij) > Uk{j), 
using j ^ i. This contradicts = (u^-i — ^ Uj.) being a strong cover and proves 
that fc > in case (m). 

Since A: > 0, we have from Lemma 13.191 

v{bi) < v{j) — n < v{j) < v{i) < v{j) + n . 

We thus have \v{i) — v{bi)\ > n, which yields 61 < i + n from Lemma [2^ Therefore, 
(a) holds. 

Finally, given that po = y{i) + kn with fc > implies y^^ipo) ~ i + kn > bi > I, 
we have that (j5.30|l follows. □ 

Lemma 5.15. In Case C, 

(5.36) u{i) > u{bi). 

Proof. By Lemma [5.131 the previous step had to be Case A or Case C, and 
the final pair {w v, y — v) it produced was commuting, by Property 15. 6r iv). 
By Lemma |3.19[ w(i) > w(bi). By Lemma |5.14[ we have either (i) 61 ^ j or (ii) 
j = bi + kn, with both implying u(j) > u(bi). Since C is a strong cover, u{i) > u{j), 
so that u{i) > u{bi) as desired. □ 

Lemma 5.16. In Case C, q and p are well- defined and satisfy 

(5.37) qo <q<P<Po 

(5.38) y-\q)<l<y-\p). 

Moreover q is A-bad and p is A' -bad. 

Proof. Given the other assertions, it is easy to see that q is A-bad from (|5.29p . 
and that p is A'-had. 
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SuBLEMMA 5.17. There is an -bad integer q such that 

(5.39) qo<q<Po 

(5.40) y-Hq)<l- 

This suffices: if q is maximal and p is the next A^-bad integer after q, then p 
satisfies by the choice of q and ((07)) holds by (jOD)) . □ 

Proof of Sublemma 15.171 Note that 

(5.41) y{i) = v{bi) < m{C^) = w{i^) = w{i) 

by Propertv l5.6f ii) applied to the previous step, and Lemma l5.13l 

By Lemma [2 . 21 applied to the strong cover y<yti^bi we have either (a) bi—i<n 
or (b) y{bi) — y{i) < n. Suppose (a) holds. We shall show that q — qq satisfies 
Sublemma 15.171 We have qo — v{i) — rt = CAw{i) — n. By Lemma 13.151 w{i) is 
A- nice. By Lemma [3.8) qo is A-bad and A^-bad. Clearly q = qo satisfies (|5.39|) . 
We have j/^^(go) — bi — n < i < I using (a) and (|5.3ip . so that q — qo satisfies 
(ICT) . 

Now suppose (b) holds. Let q' be the minimum A^-bad integer such that 
q' > 2/(i). We shall show that q — q' satisfies Sublemma 15. 171 We have 

(5.42) 90 = y{hi) -n< y{i) < q' < w{i) - 1< po < vibi) 

by assumption (b), the definition of q' , the fact that w{i) — 1 is A^-bad and greater 
or equal to y{i) by (|5.4ip . and (|5.29p . It suffices to show 

(5.43) y~\q') < I. 

If y{i) is A^-bad, then q' = y{i) and (|5.43|) follows. So we may assume that 
y{i) is not A^-bad and y{i) < q' . Since w' ^ y is a weak strip (see (|5.26p ). 
applying Lemma 13.101 to y, the A^-bad integer q' and the next smaller one, we 
obtain y~^{q') <i <l and follows. □ 

Lemma 5.18. In Case C, v'{i) < v'{bi). 

Proof. It is equivalent to show that 

(5.44) tg^py{t) <tg^py{bi). 
We have 

(5.45) y{bi) ~ n ^ qo < q < p <po < y{bi) 
by (|5.37p and (|5.29p . Using this one may deduce that 

(5.46) y(6i) < tg.p{y{bi)). 

If yii) {q,p}, then tq^py{i) = y{i) < y{bi) using (|5.3ip . If y{i) G {q,p}, then 
iq,py{i) <P< y{bi) by ^EZB- Either way ([Oil) holds by ([OS)) . □ 

Lemma 5.19. In Case C, Provertv \5.b\ holds. 

Proof. Propertv 15.6( 111) is Lemma [5. 131 
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We have m{C') ~ x{i) — CA'{u{i)) ~ CA'{m{C)), proving Propcrtv l5.6r i). For 
Propertv I5.6f ii). we must show that tq_py{i) < w{j) — 1 = po- By the proof of 
Lemma [5.181 either tq,py{i) — y{i) or tq^py{i) < p. Using either (|5.4ip or (|5.37p we 
have tq^py{i) < pq as desired. 

□ 



Lemma 5.20. In Case C, S' is a strong strip. 



Proof. We use the following notation: 



^,3 «J 
W ■ 



c- 



S[: ^ V 



i.bi 



a.-, .b-. a ,b i.b-\ 

S' : ^ y ^ v' X 

c'-- c'- C 

where {a~ ,b~) = {y~^{q),y~^{p))- We have i~ — i hy Lemma [5. 191 

By induction and the definition of S' , we need only check its last two pairs 
of consecutive marks. We have m{C') — x{i) — ca' u{i) > u{i) — 1 = Po > 
p = m{C'~), using the fact that u{i) is ^'-nice (since po q from (|5.29p and 
Lemma FS.lSp and (|5.37p . It remains to verify that m{C' ) > m{C'^^), that is, 

(5.47) p > y{a^). 

Suppose that y(aj~) < Po- Since y~^{po) > I and aj~ < / we have y{a^) < Po- Let 
r < r' be consecutive v4^-bad integers such that r < y(aj~) < r' . Since po is A^-bad 
we have r' < pq. If r = y{a^), then y^^{r) ^ <l, and thus p > q > r = y{a^) 

by the maximality of q. Otherwise, by Lemma 13.101 for the weak strip w' ~~> y we 
have y'^{r') < < I. As before r' < po. Therefore by the definition of p and q, 
p > q> r' > y{a^) as desired. 

Otherwise po < y(aj~). Since S[ is a strong strip, its last two marks satisfy 
y{a^)<yib,). By 

(5.48) yii) < wii) <po< yia^) < yih). 
By Lemma 12.21 applied to the strong cover y < we have 

(5.49) < i 
since Ui < I < bi. We claim that 

(5.50) v{j) < v{a^) < v{i). 

According to Lemma I5.14[ we have two cases to consider. First, suppose that 
Lemma [5.14f z) holds. We have y{bi) = fii,bi(6i) = v{i). Since tij is a reflection 
and bi ^ j, y[j) — v{j). Therefore ()5.48p gives v{j) < yia^) < v{i). We claim 
that aj~ ^ {i,bi}. Suppose aj~ = i. By (j5.48p y{i) < y{a^), which implies that 
i < ai , contradicting (|5.49p . Suppose — bi. Write — bi + kn. By (|5.48p 
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y{ai ) < y{bi) so fc < 0. Then since bi ^ j, we have 

= y{j) < y{a^) = y{bi) + kn< y{bi) 



n — v{i) 



But w V, w ^-s- u is a noncommuting pair so by Lemma 13.151 w(i) — w{i) < 
n, which leads to the contradiction v{i) — v{j) < n. Therefore our claim holds. 
Consequently y{a^) — v{ai) and ()5.50p holds in this case. 

Now suppose that Lemma [5IT4Jm) holds. We have y{bi) = v{i) and po = v{j) 
from ()5.35p . Since we still have v(i) — v{j) < n, we obtain y{bi) — po < n, and so 
aj" ^ holds. This leads again to (|5.50p . 

Finally, by Lemma [3.151 we know that there are no A-nice integer in the interval 
{w{i), w{j)\ and thus by Lemma l3.81 that there are no A-bad integer in the interval 
[i'(j), f («)). Letting r be the A-bad integer immediately before v{i), we have from 
(|5.50p that r < v{a~[) < v{i). Hence, from Lemma [3.101 applied to the weak strip 
w V and the consecutive A-bad integers r and v{i), that i < , contradicting 



□ 



CHAPTER 6 



Reverse Local Rule 



We describe an algorithm to compute the inverse V' = 4'u,v '■ 0° .^ X° „ of 
the local rule (f>u,v defined in the previous chapter. 

6.1. Reverse insertion at a cover 

Let C" — {v -— > x) be a marked strong cover. Reverse insertion at C is a 

A' Si 

map that takes as input an initial pair {W\Si) — {u x, u — > ■) such that 
outside(VF') = outside(C"), and produces an initial pair of the form (VF, S) — {w 
V, w — > ■), such that outside(S') — outside(5'i) and outside(W^) = inside(C"). 

There are four cases, RA, RB, RC, and RX, which denote the inverses of cases 
A, B, C, and X in the forward insertion algorithm. 

We only need to specify A, {i,j), and S, because w — c]^v and C — first(S') — 
{w ^ •)• If size(S'i) > let 

(6.1) 5i : u ^ z 

6.1.1. Commuting case. 

Case RA (Commuting Case) If (W , C) commutes, then we set A — A\ {i,j) = 
(a, 5), and S ^ C U Si. 

Example 6.1. Let n = 4,Z = and C = ([4,6,-3,3] ^ [2,8,-3,3]). 

Consider the initial pair {W',Si) where W = ([1,8,-2,3] ^-^ [2,8,-3,3]) and 
Si = ([1,8,-2,3] — > [1,8,-2,3]) has size 0. Since the pair {W',C') commutes, 
we obtain the output initial pair (W, S) where 

14^ = ([4,5,-2,3] ^-^ [4,6,-3,3]) 

and S = ([4,5,-2,3] ^ [1,8,-2,3]). 

6.1.2. Noncommuting cases. In the rest of the cases we assume that the 
pair {W , C) does not commute. By Lemma [3.191 u{b) — 1 G ^' and we set 



(6.2) A'' =A' - {u{b) - 1}. 

We say that condition B (not to be confused with case B) holds if there exists 
an ^^-nice integer q such that u{b) < q and u~^{q) < I. If condition B holds let 
qB be the minimal such q. 
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Say that condition C holds if size(S'i) > (so that is defined) and 

u{ji) is A^-niee. If condition C holds let qc = u{ji); in this case it follows that 
u{b) < u{ji) (see Lemma [6. lip . 

Note that if conditions B and C both hold then Qb ^ Qc since ji > I by the 
straddling condition. 

Case RX Suppose condition B does not hold and size(S'i) = (and so in particular 
condition C does not hold). We set w — u, A — A"^ , and S = {w — > w). 

Example 6.2. Let n = 5,Z = and C" = ([2,-5,6,9,3] [2,-5,4,11,3]). 
Consider the initial pair (T^^',^!) where 

IF' = ([2,-4,5,8,4] ^'^'^ [2,-5,4,11,3]) 

and S = ([2, -4, 5, 8, 4] — > [2, -4, 5, 8, 4]) is empty Then (11^', C") does not com- 
mute and we have = {3, 5}. Neither condition B nor condition C holds and we 

have the output initial pair is {W, S) where 11^ = ([2, -4, 5, 8, 4] ''-^^ [2, -5, 6, 9, 3]) 
and S = Si. 

Case RB Suppose condition B holds, and, in addition, either condition C does not 
hold or qb < qc- Set q = qb and let p be the maximum A^-nice integer such that 
p< q. Let A^ A'-' U {q - 1}, = {u^^ (q) , u^^ (p)) , and S ^CUSi. 

Example 6.3. Let n = 6, I — and 

C = ([4, -1, 1, 12, -3, 8] [2, -1, 1, 12, -3, 10]). 
Consider the initial pair {W',Si) given by 

11^' = ([3, 0, 1, 11, -2, 8] ^^^^^ [2, -1, 1, 12, -3, 10]) 

and Si = ([3,0,1,11,-2,8] — > [3,0,1,11,-2,8]). The pair {W',C') does not 
commute and we have — {3, 5}. Condition B holds with q = qB = 5 and p = 3. 
Thus A — {3,4,5}, = (—2,1) and the output initial pair (W, S) is given 

hyW^ ([5,0,1,9,-2,8] ^^->^^ [4,-1,1,12,-3,8]) and S = ([5,0,1,9,-2,8] ^ 
[3,0,1,11,-2,8]). 

Case RC Suppose condition C holds, and, in addition, either condition B does not 
hold or qc < qB- Set q — qc and let p be the maximum A^-nice integer such that 
p < q. We set A = U {q — 1}, (i, j) — (ii, ji), and S is obtained by inserting 
{ii, z~^(j>)) into 5*1 after the first reflection. 

Example 6.4. Let n = 4,^ = and C = (v = [4,5,-2,3] =^ [4,6,-3,3]). 
Consider the initial pair {W',Si) given by W = {u = [4,5,-2,3] [4,6,-3,3]) 
and Si = ([4,5,-2,3] =^ z = [1,8,-2,3]). The pair (11^', C") does not commute. 
We have u{b) ~ 2 and = 0. Condition B does not hold but Condition C holds 
with qc = u(l) = 4. We have p — 3, A — {3} and z^^{3) — 4 giving us the output 
initial pair (W, S) where W ^ {[3, 5, -2, 4] [4, 5, -2, 3]) and 

S = ([3,5,-2,4] ^ [1,7,-2,4] ^ [1,8,-2,3]). 
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6.2. The reverse local rule 

The reverse algorithm applied to a final pair {W , S') G 0° ^ consists of m' := 
size(S") steps, one for each cover in S'. The each step of the reverse algorithm 
is called a reverse insertion. Each reverse insertion takes as its input an initial 
pair and produces another as output. Write S' = (C(, Cj, . . . , C!^,). We initialize 
(W^(m')' '5'(m')) = {W',u — > u), where u = inside(W). For k going from m' down 
to 1, we compute the reverse insertion at on the initial pair {W(^k): '^(k)) (which 
has the property that (W(fe),C^) is a final pair), which produces an initial pair 
'5'(fc-i)) such that outside(S'(fc_i)) = ti. Let tpu,v{W,S') = {W,S,e) where 
{W,S) = (VK(o), S'(o)) and e = size(S") — sizc(5') is the number of times Case RX 
occurred. 



6.3. Proofs for the reverse insertion 

We want to compute tpu,v{W , S') with sizc(S") = m'. By induction we may 
assume that all of the reverse insertions have been performed except the last 
step, which computes the reverse insertion on (W^(m'-i), 'S'(m'_i)) at C^j/, result- 
ing in (VF(m')> '^(m'))- Again we change notation, forgetting the global meaning of 
u,v,W' , S' . We denote this last reverse insertion step as the reverse insertion on 
{W, Si) at C", resulting in (W, S). We write 

{W',C') = {u^x, V 



(6.3) 



Si : 

S: w 



C 



■ X 



a.b X 

— ^a;) 




a+,6+ 


a++,&+ + 


C' + 


C"++ 


ii Ji 
^ 






Z • • 






C+ 


Z s- • • 

C++ 



{W, C) = {w V, w u') 

In Cases RA, RB, and RC we have size(S') > so that C = first(S') is well-defined. 

In Case RX we make the convention that u' = w and write {W, 0) instead of (W, C) 

where — [w > w) is the empty strong strip going from w to itself. 

The following inductive hypothesis will be useful. In each case it must be 
re-established. 

Property 6.5. 

(i) In Case RA, m{C) = c]^}{m{C')). In Case RB, m(C) > m{C'). In Case 
RC, m(C) > c],l{m{C')). 

(ii) m(C) > x{h). 

(iii) Case RC cannot be preceded by Case RB. 

(iv) The initial pair {W, C) commutes in Cases RA and RC and does not in 
Case RB. 
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6.3.1. Case RA. By Lemma [3.141 (w u, w v) is a commuting initial 
pair such that the diagram commutes. 

a, 6 

W ^ U 

I 

A' I 

y 

V 



A' 



a.b 



X 



By Lemma [6.71 S is a strong strip. 

For the proofs below, for Case RA we specialize the general notation of (|6.3p 
as follows. 



(6.4) 
(6.5) 



S: 

S' : 



a,b il-jl 
W ^ U Z 



C 



C 



C+ 



Lemma 6.6. In Case RA, Provertv 1 6. 5\ holds. 

Proof, (iv) was proved above. For (i) we have m{C) = u{a) = c'^}{x{a)) — 

c'^}{m{C'j). For (ii) we need to show x{b) < u{a). Since v x and w > u 
are strong covers, we have x{b) < x{a) and u{b) < u{a). If u{b) is not A'-nice or 
is A'-ha,d then by (|3.4p . x{b) = CA'{u{b)) < u{b) < u{a) as desired. Otherwise by 
Lemma [331 ^{b) < u{a) as desired. □ 

Lemma 6.7. In Case RA, S is a strong strip. 

Proof. We use the notation (|6.4[) . Since Si is a strong strip by induction and 
S — C Li Si we need only show that 

(6.6) u{a) = m(C) < m(C+) = u{ji). 
It suffices to show that 

(6.7) uiji) > x(a), 

since x{a) ~ CA'(u{a)) > u{a) — 1 and a < I < ji. Since S' is a strong strip, 

(6.8) a;(a)=m(C")<m(C'+) = a;(6+). 

We apply Propertv l6.5f i) to the previous step. Suppose the previous step was Case 
RA. Then ji — fe+ and x{a) < x{ji). Since a < I < ji, (|6.6p holds by Lemma [3. Ill 
applied to the weak strip u x. Suppose the previous step was Case RB. Then 
rn(C+) > m{C'~^) > m{C') gives (|6.7p . Suppose the previous step was Case RC. 
We have 

(6.9) x{b+) ~ x{a+) < n 

by Lemma 13.191 applied to the noncommutative final pair + ) at 

the beginning of the previous step. 
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Suppose first that a = a+. Let a = a+ + kn. By (|6.8p and (|6.9p it follows that 
fc < 0. Then a;(a) = x(a+) + kn ^ x+{b+) + kn < x+{b+) < m{C^), the last step 
holding by Property 16. 5f ii1 applied to the previous step. This gives (|6.7I) . 

Suppose next that a — b^. Since a < Z < 6"*" it follows that a = b^ + kn for 
some fc < 0. Then, since ()6.9p gives a;+(a+) — < n, we have 

a;(a) = x{b+) + kn ^ x+{a+) + kn < x+{b+) + {k + l)n < x+{b+) < m(C+) , 

by Propertv l6.5t ii). 

Finally suppose that a ^ {a+,6+}. Then m{C') = x{a) = x^{a). Suppose 
x'^{a) > a;+(6+). Equality cannot hold since a < Z < 6+ so x^{a) > x^{b~^). 
By (|6.8p a;+(a+) > a;"''(a). By Lemma [3.101 for the weak strip z ^ 2:+ and the 

a+.b+ 

noncommutativity of the final pair (z ^ a; — — > 2:+), we have a+ < a. But 
this contradicts Lemma 12.21 for the strong cover x x'^ . □ 



6.3.2. Reverse noncommuting cases. By Lemma [3.201 u 
strip such that the diagram commutes. 



ti is a weak 



A^ 



A' 



a,b 



6.3.3. Case RX. In this case we have w = u and A = A so that W — {w 
v) = {u v) is a weak strip. 

6.3.4. Case RB. Thanks to Lemma [6.8[ we may apply Lemma [3.21[ which 
says that {w ^ w ^i- u) is a noncommuting initial pair such that the diagram 
commutes. 



^,3 



a,b 



A' 



This case is finished by Lemma [6.101 which shows that S' is a strong strip. 

Lemma 6.8. Suppose condition B holds. Let q = qs and p be the maximum 
A"^ -nice integer such that p < q. Then 

(6.10) Pq -.^ u{b) <p < q <u{a) + n 

(6.11) ^^{q) <l <u-^{p). 

Proof, pq is A^-nice by construction. Also M^^(po) ~ b > I. Since q > po 
it follows that p > Pq and that (|6.1ip holds. For the upper bound on q, suppose 
q > u{a) + n. Then q' := q — n > u{a) is A^-nicc. Since {W , C) does not commute, 
u{a) < u{b) and by Lemma [3. 191 these are consecutive A^-nice integers. Therefore 
q' > u{b). Also u~^{q') = u^^{q) — n < I, contradicting the minimality of q. This 
proves the upper bound in (|6.10p . □ 
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For the proofs below, for Case RB we specialize the general notation of ()6.3|) 
as follows. 

S : w ^ u 



S' 



c c+ 

a,b a+,b+ 



where = {u^^{q)TU^^{p)). 

Lemma 6.9. In Case RB, Provertv \6.5\ holds. 

Proof, (iv) was already shown, (ii) follows from (i) since m{C) > m(C') = 
x{a) > x{b), the last inequality holding because v a; is a strong cover. For (i) 
it is equivalent to show that q > CA'{u{a)). By (|6.10[) the only difference between 
A^-niceness and A'-niceness for integers in the interval [u{a), u{a) +n], is that u{b) 
is A^-nice but not A'-nice. Since {W , C) does not commute, by Lemma fS.lOi u{a) 
is A'-nice and the minimum A'-nice integer r such that r > u(a), satisfies r > u{b). 
By (|6.10p we see that q is A'-nice and q > u{h). Therefore r < q and from (|3.3p . 
CA>{u{a)) = r — 1 < 5 as desired. □ 

Lemma 6.10. In Case RB, S is a strong strip. 

Proof. By induction and the construction of S, it suffices to show that m{C) < 
m(C+). 

We have m{C) = q < u{ji) = m{C^), which follows from Lemma [6 .121, as 
u{ii) < u{ji) holds since C+ = {u ■) is a strong cover. □ 

Lemma 6.11. Suppose size(5i) > so that {ii,ji) is defined. Then 

(6.12) u{b) < x{a) < u{b+) < u{ji). 

In particular ji ^ b. 

Proof. By Lemma fS. 191 u{a) is ^'-nice and the first inequality in (|6.12p holds. 
Let S' be as in (|6.3p . Since S' is a strong strip we have 

CA'u{a) = x{a) = m(C") < m(C"+) = x{b+) ^ CA'uib+). 

Since u{a) is A'-nice and ca' is cyclically decreasing, it follows that the second 
inequality in (|6.12p holds. By Property 16.51 (ii) for the previous step we have 
u{ji) > x(b'^) > u(6+) — 1, proving the last inequality in (|6.12p . □ 

Lemma 6.12. Suppose that Si is nonempty so that (ii, ji) is defined. Letq = qB 
if Case RB holds and q = qc if Case RC holds. Let p be the maximum -nice 
integer with p < q. Then 

(1) If u(b) < u{ii) then Case RB holds and 

(6.13) u{b) <p < q <u{ii). 

(2) Otherwise we have u(ii) < u{b) < u{ji) and 

(6.14) u{b) <p < q <u{ji). 



6.3. PROOFS FOR THE REVERSE INSERTION 



49 



Moreover, whenever Case RB holds we have q < u{a) + n. 

Proof of Lemma [nHH Since first(S'i) = {u '-i^ •) is a strong cover, u(ii) < 
u{ji). Lemma 16.111 implies that either u{b) < u{ii) or u{ii) < u{b) < u{ji). 

For either (|6.13p or (|6.14p when Case RB holds, the argument that q < u{a) + n 
is the same as in the proof of Lemma 16.81 

Suppose u{b) < u{ii). Suppose there is no A^-nice integer q' such that u{b) < 

q' ^ Applying Lemma 13.101 to the weak strip u v and the A^-nice 

integer u(b) and the next larger one (which is greater than we have b < ii 

which contradicts the straddling inequality ii < I < b. So let q' be the maximum 
A^-nice integer such that u{b) < q' < u{ii). If u{ii) is A^-nice then q' — u{ii) 
and u~^{q') = ii < I. Otherwise u{ii) is not ^^-nice and q' < u{ii). Let p' be 
the maximum A^-nice integer with p' < q' . Applying Lemma 13.101 to the weak 

strip u V and consecutive ^^-nice integers p' < q' , we have u~^{q') < ii < I. 
Therefore Case B holds. 

Suppose u{ii) < u(b) < u{ji). Let q' be the maximum A^-nice integer with 
<?' < ""(ji); it exists and satisfies u{ii) < q' since u{b) is ^^-nice. 

If q' — u{ji) then either Case RB with q < q' will hold or Case RC will hold 
for q = q' . ((67T4ll follows. 

Now suppose q' < u{ji). By Lemma [3.101 applied to W' , we have u'^{q') < ji. 
But by Lemma [2.21 applied to first(S'i) we must have u^^{q') < ii < I. Therefore 
q' is a witness that Condition B holds. (2) follows. □ 

6.3.5. Case RC. Again we sketch the proof and then fill in the Lemmata 
which prove the details. 

By Lemma 13.201 u v is a weak strip such that the diagram commutes: 



U s- z 



A' 



a, 6 



By Lemma 16.131 (u ^ w, u z) is a commuting initial pair. Let x' — vti^j^ = 

CAvz. By Lemma 13.141 (z ~-+ x' , v x') is a commuting final pair such that the 
diagram commutes. 




Recall that qc = u{ji) = z{ii). Let — {ii,z ^ip)) and u' = zti+ j+ — 

tp_q^z. By Lemma 16.141 we may apply Lemma 13.211 to the weak strip z u and 
pair p < qc oi consecutive ^^-nice integers, so that (u' ~^ a;', u' — ^ z) is a 
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noncommuting initial pair such that the diagram commutes. 




By Lemma 16.151 the final pair (u' x, v -^-^ x) commutes. Define w = u'ti-^j-^ = 
c^^(w). By Lemma [3.141 (w v, w u') is a commuting initial pair such that 
the diagram commutes. 



A 




In particular, as defined, is a strong tuple and is a weak strip. Lemma 16.171 
shows that S' is a strong strip. 

Lemma 6.13. In Case RC, v{ii) < v{ji). 

Proof. Since u z is a strong cover, u{ii) < u{ji). Now v — ca-^u and 
u{ji) is A^-nice since Condition C holds. It follows that v{ii) < v{ji). □ 



Lemma 6.14. In Case RC, z ^(p) > I. 



Proof. Suppose not, that is, < I- By Lemma [6. 12) uih) < u{ji) — qc- 

Now p < qc are consecutive A^-nice integers. Since u{b) is also ^^-nice it follows 
that u{b) < p. Suppose u{b) = p. Since z~^{p) = ti-^j-^u~^{p) = u~^{p) given that 
^ < n and u{ii) < p < u{ji) from Lemma 13.151 on the non-commuting 

initial pair {u ^ x, u z), we have the contradiction I < b = u^^{u{b)) = 
"""^(p) = z^^{p) < l- So u{b) < p. But then condition B is satisfied by the integer 
p which is less than qc, meaning that Case RB holds, which is a contradiction. □ 



Lemma 6.15. In Case RC, u'(ii) > u'(ji). 



Proof. Since u'(ii) = tp^q^z{ii) = tp,q^{qc) = p, and u'(ji) = tp,q^z{ji) = 
tp.qcu{ii), this is equivalent to 

(6.15) p > tp^q^u{ii). 

Since we are in Case RC, by Lemma 16.121 we have u{ii) < u{b) < p < qc ~ ""(ji)- 
If it(ii) ^ {p,qc} then (|6.15p clearly holds. Otherwise u{ii) e {p + kn,qc + kn} 
for some integer fc < 0. Then tp^q^u{ii) G {p + kn, qc + kn}. Then (|6.15p follows 
from p + kn < qc + kn < qc — n < p, the last inequality holding since p and qc are 
consecutive A^-nice integers. □ 
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For the rest of the proofs we use the following notation for Case RC. 



C c"+ c 

(6.16) s, : 

S: 

where = z^^(p). 



Lemma 6.16. Property 1 6. 5\ holds in Case RC. 



Proof. We use the notation (|6.16p . Since ni{C) = u'{ii) = p, as seen above 
equation (|6.15[) . (i) is equivalent to p > u{b) which holds by Lemma [6. 121 (ii) then 

follows since the non-commutativity of the final pair {u ^ v, u > x) implies from 
Lemma [3. 191 that u{b) is not ^'-nice, and thus that x{b) — c^;u(6) = u{b) — 1 < p 
from (|3.3p . (iv) was shown above. For (iii), suppose Case RC was preceded by Case 
RB. We use the following diagram. 



V 




+ 



A' ah 1 + A'^ 

By assumption the final pairs {W , C) — {u x, v ^^-^ x) and (W^ , C" ) — {z 

a+.b+ 

+ ) both do not commute. 
We will see that it suffices to show that 

(6.17) There is an integer i' < I such that u{b) < u{i') < u{ji). 

Let i' be as in ()6.17|) . Then u{i') is not A^-nice; otherwise Condition B would 
hold for u{i'). Let r < r' be the pair of consecutive A^-nice integers such that 
r < u{i') < r' . Since u{b) and u(ji) are A^-nice (the latter by the Case RC 
assumption) we have u{b) < r < u{i') < r' < u{ji). By Lemma 13.101 for the weak 

strip u V we have u^^(r) < i' <l. This gives the contradiction that Condition 
B holds for r if u{b) < r. But if u{b) = r then I < b = u~^{u{b)) = u~^{r) < I, a 
contradiction. 

We now prove (|6.17p . Since S' is a strong strip, 

(6.18) x{b) < x{a) < x{b+). 

Suppose a a^. Wc claim that (|6.17[) holds for i' — a^. By Property [631^ ii) for the 
previous step, we have u{ji) > x^{b'^) = x{a~^) > u{a^) — 1, that is, w(ji) > w(a+). 
But a'^ < I < ji so u{ji) > u(a+), giving the right hand inequality in (|6.17p for 
i' — a^. We are done if u{b) < u(a+). Suppose not. Since a'^ < I < b we must 
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have u{b) > u(a+). We claim that 

(6.19) x{a+) < x{h) < x{a) < x{b+). 

Since {W',C') is a noncommuting final pair, by Lemma 13.191 u{a) is A'-nice, u{b) 
is not A'-nice, and x{a) > u{b) > u{b) — 1 = x{b). If u{a) < u(a+) then u(a+) 
is not ^'-nice and x(a+) = w(a+) — 1 < u{b) — 1 = x{b) so that (|6.19p holds. If 
w(a+) < u{a) then since u{a) is A'-nice, x(a+) < u{a) < x{b) and again (|6.19p 
holds. By Lemma 12.21 applied to the strong cover C'^ we have a < a+. Now 
< a;(6+) — a;(a+) < n by Lemma r3.19l for the noncommuting final pair {W'^ , C'^). 
Therefore (pTTg)) gives < a;+(fe) < x+{a) < x+{a+). By Lemma [5TTU1 for the 

weak strip W'^ we have a+ < a, a contradiction. 

Suppose a — a^. We claim that (|6.17p holds for i' = ii. Since u z is a 
strong cover we have u{ii) < the right hand inequality in (j6.17p for i' — ii. 

It suffices to show that 

u{ii) = z{ji) > z{b+) > z{b+) - 1 = x+{b+) = x{a+) = x{a) > u{b). 

The first inequality holds by (|6.13p for the previous (Case RB) step. The second 
equality holds by Lemma 13.191 for the final pair (VF+,C"+). The last inequality 
holds by Lemma [3Tl9] for the noncommutative final pair {W , C). □ 

Lemma 6.17. In Case RC, S is a strong strip. 

Proof. By the construction of S and the fact that Si is a strong strip by 
induction, we need only show that 

m(C) < m(C+) < m(C++). 

The second inequality holds because m(C+) = z{i^) = z{ii) = qc = m(first(5i)) < 
m(C^+), the inequality holding by the strong strip condition of Si. Given that 
m{C) — p, the first inequality holds since p < qc follows from Lemma 16.121 □ 



CHAPTER 7 



Bijectivity 

Our main theorem is the foUowing. 

Theorem 7.1. The maps tjj and <j) are inverses to each other. Thus the map (j) 
is a bijection. 

Our approach to proving bijectivity is to reduce to the case of at most two 
steps, exploiting the fact that the maps cf) and ip can be "factorized" into "smaller" 
instances of (j) and ip^ to which induction may be applied. 

Consider the sequence of steps involved in computing (f>{W, S, e). With respect 
to this sequence, we call a subsequence of consecutive steps irreducible if it consists 
of: 

(1) a Case X step. 

(2) a Case A step followed by some maximum number m (possibly zero) of 
consecutive Case C steps. 

(3) a Case B step. 

Mnemonically we denote such irreducible sequences by X, A{m), and B respectively. 

Dually, consider the sequence of steps involved in computing tp{W', S'). With 
respect to this sequence, we call a subsequence of consecutive steps irreducible if it 
consists of: 

(1) a Case RX step. 

(2) a Case RA step followed by some maximum number m (possibly zero) of 
Case RC steps. 

(3) a Case RB step. 

Denote these mnemonically by X~^, A{m)~^ and respectively. As a warning, 
note that, for example, when m = 2 the inverse of Case A followed by two Case 
Cs, is Case RA followed by two Case RCs. 

It is clear that every sequence has a unique factorization into irreducible sub- 
sequences. 

We shall show in Section \7A\ that if the forward algorithm ends with X then 
■0 o = id and if the reverse algorithm begins with X~^ then o ^ = id. 

In Section 17.21 we show that if the forward algorithm starts with .4(0) or the 
reverse algorithm ends with yl~^(0) then we have bijectivity. 

In Section 17.31 we show bijectivity when the forward algorithm begins with B 
and when the reverse algorithm ends with B^^. 
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In Section r7.4l we show bijectivity when the forward algorithm begins with A{m) 
and when the reverse algorithm ends with A{m)^^ for m > 0. This is accomplished 
by reducing to the case that m = 1. 

This covers all cases, so Theorem 17. II follows. 

In each of the following sections, for the proof of ^ o (/> = id, we suppose that 
{W, S, e) e 1°^ and denote {W , S') — (j){W, S, e) and S for the sequence of steps 
in this computation. For the proof of (j) o ip = id we suppose {W , S') G 0° write 
{W,S,e) = ip{W',S') and by abuse of notation write to denote the sequence 
of reverse insertion steps in the computation of {W, S, e). 



For ° 01 suppose S has the form S — SiX, that is, it ends with X. The last 
(Case X) step depends only on the weak strip entering that step, and Si is itself 
a "smaller" instance of cj). By induction we may reduce to the case that S = X 
consists of a single Case X step. 

For o -0, suppose S^^ has the form S^^ = X^^S^^ . The first step, which 
is Case RX, depends only on the last cover in 5". The second step will not be 
Case RC, so iSj~^ is a smaller instance of tp. By induction we may assume that 
S^^ = X~^, so that S' consists of a single cover C . 

So for "00 0, let {W, S) be a final pair such that W — {w v) with < n — 1 
and S ~ (v — > v) ~ the empty strong strip. Let W i— > {W',C') be the 

result of external insertion with {W',C') — {u ^ x, v x), A' — AU {q} and 
(a, 6) = {v~^{q),v~^{p)) where q < p are the pair of consecutive A-bad integers 
such that v^^{q) < I and q is maximal. By construction the final pair (W',C') is 
noncommuting. 



Applying the first step of tp to {W',C'), we cannot be in Cases RA or RC. By 
Lemma |3.15[ u(b) is not A'-nice and thus u{b) — 1 = CA'{u{b)) ~ x{b) ~ v{a) = q. 
Therefore A^ as defined in (|6.2p . equals our A. By Lemma [3.8[ u{q') is A-nicc if 
and only if v{q') = CA{u{q')) is v4-bad. So condition B fails due to the maximality of 
q in the external insertion. Therefore Case RX occurs and it produces the original 
weak strip W as desired. 

For (j)oip, let S' = C" be a single strong cover and {W , C) — {u ^ x, v x) 
a final pair such that Case RX occurs. In particular {W',C') does not commute. 
We define A = A^ as in (|6.2p and have the commutative diagram with weak strip 



7.1. External insertion 



u 




V ^ X 

a, 6 



7.3. CASE B (BUMPING CASE): 
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W = {u v). 



By Lemma [3.191 uia] < u{h) are consecutive ^^-nice integers and u{b) is not A'- 
nice. We now apply external insertion. Let q < p he consecutive A^-bad integers 
such that v^^{q) < I with q maximum. We have u{b) — 1 = x(b) = v{a) so it suffices 
to show that q = v{a) and p — v{b). By Lemma 13.81 v(a) < v{b) are consecutive 
A — A^-bad integers and v~^{v{a)) — a < I. If there was an A^-bad integer 
q' > v{a) with v^^{q') < I, then q' > v{b) and c^i(g') is an A^-nice integer greater 
than u{b) with ii^^(c^i (q')) — v^^{q') < I, so that condition B holds, contradicting 
the assumption that Case RX occurs. Therefore q — v{a) and p = v{b) as desired. 

7.2. Case A (commuting case) 

In the case that S (resp. S^^) consists of a single Case A (resp. RA) step, 
bijectivity holds by Lemma [3. 141 

In general, for -0 ° 0: suppose S starts with a Case A step which is not followed 
by a Case C step. Write S = ^(O)iSi. Since the first step of Si is not Case C 
by assumption, iSi is an instance of (/) involving fewer steps. By induction we may 
assume that 5 is a single Case A step, since a single Case RA step is unaffected by 
the strong covers that were produced previously. 

For (j) o we may similarly reduce to the single RA step case. 

7.3. Case B (bumping case): 

For ip o (/) we first suppose that the entire sequence 5 is a single Case B step. 
Let {W, C) = (w V, w u) be a noncommuting initial pair. As in Case B we 
define — A—{u{i) — 1} and let q < p he the pair of consecutive yl^-nice integers 
such that q < u{j) and u^^{q) < I, with q maximum. Defining A' = U {p — 1}, 

{a,b) — {u^^{q),u^^{p)) and x — vta.b — ca'U, we have that u ^ v is a weak 

strip and (W , C) = (u x, v x) is a noncommuting final pair such that the 
diagram commutes: 

W ^ U 

/ 1^' 
> Y 
V ^ X 

a,b 

We now apply the reverse algorithm to {W',C'). In the single cover context 
Case RC does not occur, and (W, C") does not commute. By Lemma IB.igi Mfb) = p 
is not ^'-nice. Then A' - {u{b) - 1} = A' - {p - 1} = A"^ so that the above 
definition of A'^ agrees with the one in the noncommutative case of the reverse 
algorithm. Now q — u(a) satisfies q < u{j) by its definition, so u{a) < u{j). Since 
{W, C) is a noncommuting final pair, by Lemma [3.19l M(a) < u{b) are consecutive 
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A^-nice integers. Since {W, C) is a noncomniuting initial pair, by Lemma 13.151 
u{i) < u{i) are consecutive ^^-nice integers. Therefore we have u{a) < u(h) < 
u{j) < u(i). In particular condition B holds with the A^-nice integer u{i). So Case 
RB holds, and say it selects the consecutive A^-nice integers u{b) < pb < Qb < u[i). 
If qB < u(i) then since u{j) < u{i) are consecutive A^-nice integers and j > I, 
it follows that u{b) < qb < contradicting the maximality of q in Case B. 

Therefore qB — u{i) and pB ~ ^(j). It is now clear that ip o (p = id. 

We now return to the general case of V' ° 0- Let S = { w s- u ^ ■ ■ • )• 

We know that the output of the first (Case B) step is a noncommuting final pair. By 
Propertv l5.6f iii) the second step of (j) is not Case C. By induction we suppose that 
performing the rest of (j) and then all of tp except for the last step, is the identity. 
We now consider the last step of the reverse algorithm -0; it is the noncommutative 
case. It suffices to show that Case RB occurs in this last step. By the single cover 
case we know that condition B holds with qb = u{i), so that Case RX cannot 
occur. Since S" is a strong strip we have u{i) < u(ji) so that Case RC cannot hold. 
Therefore Case RB holds and we have reduced to the single cover case. 

The reasoning for o ^ in the single cover case, is entirely similar to that for 
tpocj) 'm the single cover case. Suppose now that the last step of ip on {W, S') G 0° 
is Case RB. After applying ip to {W , S') we apply (f>. Its first step is Case B and 
undoes the last step of ip by the single cover case. The second step of 4> cannot be 
Case C, and by induction the rest of (p is the inverse of the rest of ip, and we are 
done. 



7.4. Case C (replacement bump) 

7.4.1. Reduction to m — 1. For ip o cp suppose S = A{m)Si with m > 
maximal. Then Si does not start with Case RC and the output strong cover of the 
last (Case C) step oi A{m) is involved in a commuting final pair by Propertv l5.6r iv'). 
It follows that the usual reduction works and wc may assume that S — A{m). 

For (poip suppose S^^ — Si^A{m)^^ for m > 0, that is, ip produces a sequence 
of steps that ends with Case RA followed by some positive number of Case RC 
steps. Since is followed by a Case RA step, it is a smaller instance of ip. In 
particular its inverse cannot begin with Case C. Since Case RA steps are unaffected 
by previously produced strong covers, again by induction we may assume that 
S = A{m). 

Let us consider ip o (p on {W, S, 0) such that (p on (W, S, 0) consists of a single 
Case A step followed by m Case C steps for some to > 0. By Section FTT] the output 
{W, S') is such that ip on {W, S') does not start with RX. We shaU reduce to the 
case TO = 1 by showing that (p can be achieved by splicing together two operations: 
the first two steps of (p, which consists of Case A followed by a Case C, and another 
application of (p which consists of a Case A step (the "second half of the first Case 
C step) followed by to — 1 Case C steps. 
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Suppose m > 1. Let 



Uq ^ Ul s 



Since S' is a strong strip we have 
(7.1) uo{i) < -uo(ii) 



Ui{l) < Ui[j2). 



Let W = {w v) and 0(14^,5,0) = (VF',S"); since this is a Case A step followed 
by m Case C steps, S' has the form 

S' = [t;; ((ai,6i), (02,62), . . . , (a™,6„), (i, j));a;]. 

The computation of (piW, S, 0) starts with a two step computation 

cj>{W, [w- (i, ji)); Ul], 0) = (Wi, [v; ((ai, 61), (z, j)); xi]), 

where Wi = (ui ^ xi). By Property [5TD(iv) apphed to the Case C step, the final 
pair {Wi, xi tij xi) commutes. By Lemma 13.141 (ui ty ^ xi tij, ui tij ui) 
is a commuting initial pair such that the diagram commutes. 



Ml t 



1 



Ml 



A* 



1 



Xl 



Let 



A" 



(mi tij j^j^ i/^j ^ 

[Mitjj-; ((i,j), (i,j2) 

Due to (|7.ip we see that (VF*, 5**) is an initial pair. It is clear from the definitions 
that 

4>{W\S*) = [xi Uj- iia^M). . . . , (anx, h^), (z, j)); x]) 
which is a Case A step followed by m — 1 Case C steps. By induction we have 
o 0(W^*, S"*) = (M^*, S"*). Thus we have reduced to the case that m = 1. 
In the case that "0 on (VF', S") £ 0° „ consists of an RA step followed by m > 1 
RC steps, we may apply a similar reduction to the m ~ \ case. 



1 A.I. m — 1. So we assume m = 1. For ip o (j), we start with the initial pair 
{W, S) where 

W — {w^ -A y) 

S = [w^ ^ W ^ M ). 
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We give diagrams before and after the Case C step. 

yj- 5- W ^ U w' ^ U 




A' 



y ^ V y ^ y' ^ X 

»J a.b i,j 

Here A"^ is defined by dEH]) and A' ^ U {q} where (a, b) = {y^'^iq),y^'^{p)) and 
q < p is the consecutive pair of A^-bad integers such that q < u{i)~l and u^^{q) < I 

with q maximum. We have 5" = ( 2/ — ^ v' — x ) and W = {u ^ x). By 

Propertv IS.Gf iv). the final pair {u ^ x, v' — ^ a;) commutes. We now apply ^p. 
The first step is the commuting step RA, which produces the commuting initial pair 
[w' ^ u', w' u). We now apply the next reverse insertion step. By the Case 

C construction, the final pair [w' v' , y v') is noncommuting. In particular 
by Lemma fB.lQ) w'{b) is not A'-nice and 

(7.2) w'{b) - 1 = CA'w'ib) = v'{b) = y{a) ^ q . 
In this situation the reverse insertion algorithm defines 

A"" = A' - {w'ib) - 1} = A' - {q} 

which agrees with as defined above. 

Condition C holds since w'{j) — u{i) is y4^-nice by definition. If Case C holds, 
then it is easy to verify bijectivity: we have qc — w'{j) = u{i), the previous ^^-nice 

integer is indeed pc = u{j') by Lemma 13.151 for the initial pair {w v, w — ^ u) 
for the forward direction, and the definition of A for the current RC step agrees 
with the above definition of A, as both are obtained from A"^ by adding the element 
qc - I ^ u{i) - 1. 

So it suffices to show that Case C holds, that is, there is no A^-nice integer 
qB such that w'{b) < qs < w'{j) and g :— w'^^(qB) < I- Suppose such an integer 
exists. We must derive a contradiction. 

We have 

(7.3) po = u{t)^l = w'ij)-l. 

We claim that q' = y{g) contradicts the definition of q in Case C. By definition 
y~^{<l') =9^1- We have that y{g) = CA^{w'(g)) — CA-^iqs)- Since q < qs — I < 
u{i) — 1 = pq are all ^^-bad integers, it follows that qs — I < y{g) are consecutive 
v4^-bad integers and that y{g) < Pq. Furthermore y{g) < Po = — 1 since 
9 < I < y~^{Po) by (|5.30p . It only remains to show that y{g) > q. We have 
y{g) ^ {q,p}, by Lemma [5. 161 and the fact that g < I < y~^{p)- Therefore 

2/(5) = tqpvig) = tqpCA-^iqs) ^ CA'iqs) > q + i > q 

and we arrive at the desired contradiction. 

For o -0, the argument is nearly the same. By the definition of Case RC, it is 
clear that in the subsequent calculation of 0, Case C will be invoked at the second 
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step. The only thing which needs to be checked is the maximahty of q in Case C, 
which follows from the fact that Case B does not hold in the second step of tjj. 



CHAPTER 8 



Grassmannian Elements, Cores, and Bounded 

Partitions 

Let k = n — 1 from now on. The fc-Schur functions, denoted s^'' (x) for u e 
in Theorem 14. Hi are traditionally written sl^'^'' (a;) where A is a partition such that 
Ai < k. Weak tableaux for Grassmannian elements were first introduced |20j as 
/c-tableaux, which are defined in terms of (fc + l)-cores. In this chapter we recall 
bijections between Grassmannian elements, offset sequences, cores, and bounded 
partitions. 

Let ^ = in this chapter. The set = S"^ of Grassmannian elements is defined 
in Section [221 

8.1. Translation elements 

Let Q be the coroot lattice of s[„, realized as the set of n-tuples of integers with 
sum zero. Sn acts on Q by permuting coordinates. Given /? = . . . ,/3n) G Q, 
the translation element Tp G Sn is uniquely defined by T^(i) = i + n(3i for 1 < i < n. 
We have Tf^Tj = so that T[Q) = {jp \ (3 G Q} forms an abelian subgroup of 

Sn isomorphic to Q. By (|2.ip we have 

(8.1) £{Tp)^ J2 \^i-f^^\ 

l<i<j<n 

from which it follows that 

(8.2) £{Tp) = £{tup) for /? G Q and u G Sn 

11 

(8.3) t{Ti3) = 2 ^ i/3i = 2(p' , P) for /3 antidominant, 

where p' = (1, 2, . . . , n) and an antidominant element is one that is weakly increas- 
ing. The inner product (. , .) is the standard one on M". 

T{Q) acts on Q by translations: 173(7) = /? + 7 for all 7 G Q. Sn acts on 
T[Q) by conjugation: SiTpSi — Tg.p for 1 < i < n ~ I and /3 G Q. There is a well- 
known isomorphism ( [121 Prop. 6.5]) Sn — Sn x T{Q) under which sq ^-^ sgT-e 
where 6 — (1, 0, . . . , 0, — 1) G Q is the highest coroot and sg is the associated 
reflection, which satisfies sg = ti n = S1S2 ■ ■ ■ s„_2Sn-iSn-2 • • • S2S1 and acts on Z" 
by se(ai, . . . ,a„) = (a„,a2, . . . ,a„_i,ai). 

Again using (|2.ip . for v G Sn and 7 G Q we have 

(8.4) £{vT^) = £{t^) + J2 Xiv{t) > vim^ir^-''^''^^ 

l<i<j<n 

61 



62 



8. GRASSMANNIAN ELEMENTS, CORES, AND BOUNDED PARTITIONS 



where x{S) = 1 if 5 is true and x{S) = if 5 is false. 
Example 8.1. Let n — A and 

w = S1S2S3S0S3S2S1S0S3S2S0S3S1S0 = [-7, -1,4, 14]. 
Then WS3S2 — [—7, 14, —1,4]. In we have the equahty 

(-7,14,-1,4) = (1,2,3, 4) + 4(-2, 3,-1,0), 
so WS3S2 = <-2,3,i,o and w = t-2.3,-i,oS2S3 = 5253^-2,-1,0,3- By ()8.4p we have 



e{w) = -2 + £(t_2,-i,o,3) = -2 + 2(l(-2) + 2(-l) + 3(0) + 4(3)) = 14, matching 
the length of the above reduced word. 

For /3 e Q let S^^ C Sn denote its stabilizer. 

Proposition 8.2. Let u e Sn- 

(1) If (3 G Q is antidominant and u is of minimum length in its coset uS!^ 
then 

(8.5) £{uTp) = £{Tp) - e{u). 

(2) Let u and (3 he as in {I]). 1/7 G Q and v G Sn are such that vj — uf3 then 
i{vTy) > i^uTff) with equality if and only if v — u and 7 = /3. 

Proof. Equation (|8.4p says that £{vt^) — 1{t^) is the number of inversions {i,j) 
of V such that 7; > 7^ , minus the number of inversions of v such that 7.^ < 7^ . For 
u and [3 in the hypotheses, we have (3i < (3j and (3j — Pi implies u{i) < u{j). This 
is precisely the condition that the first set of inversions is empty and the second is 
the set of all inversions of u. This proves (H]). 

For ^ we must show that for {v, 7) e S'„ x (S'„ • (3) such that v-f — u/3, £{vTj) 
is uniquely minimized by the pair (u,j3). 

First let 7 6 S'n • /? be fixed. Suppose for some 1 < p < q < n, "fp = and 
v{p) < v{q). Let v' = vtp,q. We shall show that i{vTj) < £{v'tj), which implies that 
the desired minimum clement (^,7) must have the property that v is of minimum 
length in the coset vS^- For 1 < i < j < n let 

(8.6) f{i,j) = i-l)^^'"<'^'Hxiv'{t) > v'm - x{v{i) > v{j))) 
so that using (|8.4p 

(8.7) £{v't,)-£{vt,)= 

l<i<j<ji 

For (i,j) such that {i,j} H {p,q} — SZS, f{i,j) = 0. For {i,j) such that i — p and 
j ^ q we have p < j and 

f{p,j) = {-ir^''-<''^Hx{v{q)>v{j))-x{v{p)>v{j))) 

- (-l)x(><^^)x(^'(p) < v{j) < v{q)) 
For {i,j) such that i ^ q we have q < j and 

/(9,j) = (-i)^(^'<''^-Hx(«(p) > t'(j-)) - x(«('z) > vm 

= -(-1)^^^''<^^-^X(«(P) < v{j) < v{q)) 
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using 7g = 7p. We have 

j>p j>q p<j<q 

(8.10) 

= E (-l)''^^''^'''^x("b) < v{j) < v{q)). 
p<j<q 

Similarly 

(8.11) ^ f{^,p) + ^ /(z, 9) = 5] (-l)'^(^'<>)x(t'(p) < ^^(0 < v{q)). 

i<p i<q p<i<q 

The remaining term is (i, j) — (ja, q): 

(8.12) /(p, g) = (-l)x(><>)(x(^;(g) > v{p)) - x(«(p) > ^^(g))) = 1- 
Combining (|gTTU)l . and we have 

(8.13) i{v'T^) ~ l{vT^) = 1 + 2 ^ x(7p = 7.)x(t'(p) < ^^(0 < v{<l)) > 1- 

p<i<i3 

We may therefore assume that v is of minimum length in its coset vS^. Suppose 
next that 7 7^ /3, so that there is an index r such that 7^ > 7r+i- Let 7' = 5^7 and 
v' — vsr so that v'j' = U7. It suffices to show that 1{v't^i) < 1{vt^). Let 

(8.14) 5(*,j) = {-lY^<<-<'^h{v'{^) > v'ij)) - (-l)^(^'<^^)x(«(*) > v{j)) 
Due to and we have 

(8.15) £(«V)-^(^'T^)= E dihj)- 

l<i<j<n 

If {«, j} n {r, r + 1} = then g{i, j) = 0. For i ~ r and > r + 1 we have 

(8.16) g{r,j) = (-l)x(^-+i<^^)x(«(^ + 1) > v{j)) - (-l)'^(^'-<^^-)x(«(0 > 
For i = r + 1 and 7 > r + 1 we have 

(8.17) g{r+l,j) = {-l)^^^"'<^^^x{v{r) > v{j))-{-l)^^^^+'<^^^x{v{r+l) > v{j))) 

These cancel: for j > r + 1 we have g{r,j) + g{r + 1, j) = 0. Similarly for i < r we 
have g{i^ r) + g{i^ r + 1) = 0. For — {r,r -\- 1) we have 

g(r, r + 1) = {-l)^'^^^+^<^^\{v{r + 1) > i;(r)) 

= — x(^(^ + 1) > '^C'^)) ~ x(''^(^) > '^(^ + 1)) 
= -1 

so that £{v'T-y') — £{vTy) — 1, which suffices. □ 

8.2. The action of Sn on partitions 

Consider the positive quadrant Z?,q in the plane, where an element (j,j) is 
depicted as a cell (square) in the plane, indexed using standard Cartesian co- 
ordinates. The diagram of the partition A = (Ai, A2, . . . , Ap) is the set of cells 
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{{hi) Ai} with Xi left-justified cells in row i for all i. De- 

fine the diagonal index of a cell by diag(i, j) = j — i and the residue of a cell by 
res(i, j) = j — i d TLjnl. 

Given a partition A, one may associate a bi-infinite binary word p(A) = p = 
■ ■ -p-ipopi ■ ■ ■ called its edge sequence. The edge sequence p( A) traces the border of 
the diagram of A, going from northwest to southeast, such that every letter (resp. 
1) represents a south (resp. east) step, such that some cell in the i-th diagonal is 
touched by the steps pi-i and pi. 

Example 8.3. The diagram of A = (10, 7, 4, 3, 2, 1, 1, 1) is pictured below. 



■ 



The edge sequence isp(A) = • • • 11|0111|0101»0100|0100|0100| ■ • ■ where • indicates 
the diagonal, which separates the bits p_i and po. 

The affine symmetric group 5„ acts on partitions in an obvious way, if we iden- 
tify elements of Sn with functions Z ^ Z and partitions with their edge sequences, 
which are certain functions Z {0,1}. Say that the cell x is X-addable (resp. 
X-removable) if adding (resp. removing) the cell x to (resp. from) the diagram of 
A results in the diagram of a partition. Then for i G Z/nZ, SiX is obtained by re- 
moving from A every A- removable cell of residue i, and adding to A every A-addable 
cell of residue i. 

Example 8.4. For n = 4, applying the reflections of the reduced word of 
w in Example 18.11 to the empty partition from right to left, we obtain the se- 
quence of partitions () C (1) C (2) C (2,1) C (2,2) C (3,2,1) C (4,2,2) C 
(5,2,2) c (6,3,2,1) c (7,4,2,2) c (8,5,2,2) c (9,6,3,2,1) c (9,6,3,3,1,1) c 
(9,6,3, 3,1, l,l)c (10,7,4,3,2,1,1,1). 

In fact, the action of Sn on partitions coincides with the action of the Kashiwara 
reflection operators on the crystal graph of Fock space [30j . 

8.3. Cores and the coroot lattice 

An n-ribbon is a skew partition diagram A//i (the difference of the diagrams of 
the partitions A and /i) consisting of n rookwisc connected cells, all with distinct 
residues. We say that this ribbon is X-removable and ^-addable. An n-core is a 
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partition that admits no removable rt-ribbon. Henceforth when we say "core" we 
mean "n-core" . Since the removal of an n-ribbon is the same thing as exchanging 
bits Pi = and = 1 in the edge sequence for some i, it follows that A is a core if 
and only if for every i, the sequence p'^'^(A) :— ■ ■ ■ Pi-2nPi-nPiPi+nPi+2n ■ ■ ■ consist- 
ing of the subsequence of bits indexed by i mod n, has the form • • ■ 1111100000 • ■ • . 
Thus the core is specified by the positions where this sequence changes from 1 to 
for various i. For a core A and i ^ Z let di{X) = e Z be the integer such that 

= 00 The sequence {di)i^i is called the extended 
offset sequence of A, and d{X) — (di, ^2, . . . , dn) the offset sequence of A. Observe 
that 

(8.19) d,_„ =di + l for alH e Z 

and that X]r=i '^^ ~ 

Example 8.5. For n = 4 and A = (10,7,4,3,2,1,1,1), we have d{\) = 
(—2,3,-1,0). These are the heights "above sea level" that the Is attain if we 
draw the edge sequence p{\) in an oo x n array with the r-th row given by the 
binary word with bits Pi+nr for i = 0, 1, 2, . . . , n — 1, and "sea level" is the division 
between rows —1 and 0. Below we depict the bit sequence p(A), whose columns, 
read from bottom to top, arep^°^(A) through p^"^^^( A). 
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The following results are well known; see for example [241 127) . 

Lemma 8.6. For any core A and i G IjnlL, either there are no X-addable cells 
of residue i or there are no X-removable cells of residue i. 

Proposition 8.7. There is a bijection from the set Cn of n-cores to the root 
lattice Q of sin given by X t-^ d(A). 

Q already has an action of 5'„ defined in Section 18.11 which is transitive since 
the subgroup T{Q) of S'„ acts transitively on Q. Via the bijection A i-^ d{X), Sn 
acts transitively on the set C„ of cores. This induced action coincides with the 
action of Sn on partitions defined above. 



Using i — 1 rather than i allows an elegant statement of Proposition [93] 
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Proposition 8.8. The action of Sn on partitions restricts to an action onC„. 
Moreover, the bijection of Proposition \877\ is an isomorphism of sets with Sn-action: 

(8.20) d{w ■ X) = w ■ d{\) for every w G S'„. 

Proof. It suffices to consider the case w — s^. This is easily verified, the most 
interesting case being i — 0, where one uses (|8.19p . □ 

Proposition 8.9. C„ — Sn ■ ■ In particular there is a bijection 

(8.21) c : Si Sn/Sn Cn 

(8.22) w ^ wSn t-^ w ■ 0. 

Proof. We already know that Sn acts transitively on C„ and £ C„, so that 
Cn — Sn ■ 0. Since Sn is the stabilizer of the result follows. □ 

8.4. Grassmannian elements and the coroot lattice 

The bijection doc : S^ ^ Q has the following affine Lie-theoretic interpretation. 

Proposition 8.10. Write w e S" as w — urp where u e Sn and f3 ^ Q. 

(1) d{c{w)) =upeQ. 

(2) (3 is antidominant and u is of minimum length in uS^ where S^ is defined 
before Provosition \8.2l 

(3) e{w)=£{tp)^£{u). 

Proof. Let A = c{w) (that is, A = w • 0) and d = d{\). We have d{c{w)) = 
d = d{w • 0) = w • d{0) = w ■ (0") = uTp{Q") = u(3 using Proposition EUl In fact 
this works for any w G 5„ such that w ■ = X and w = mt^. 

Suppose w' = VT~^ is such that w' ■ ~ X. Then vj = d as well. We have 

w' = VT^ = wr„-i„0 

= VV~^UTl3U~^V 

= UTp{u^^v) = w{u~^v). 

The result follows from Proposition 18. 21 □ 

Example 8.11. For n = 4 and w = [-7,-1,4,14], by Example 18.11 we have 
u = S2S3 and (3 ~ (—2,-1,0,3). So d = u(3 ~ (-2,3,-1,0), which agrees with 
Example 18.51 

8.5. Bijection from cores to bounded partitions 

Say that a partition A is n-boundec|^ if Ai < n. Denote by B„ the set of 
n-bounded partitions. 

The hook length of a cell {i,j) in a skew shape A//i is the number of cells in 
A//i in the i-th row to the right of {i,i), plus the number of cells in X/ pi in the j-th 
column above (i, j), plus one (for the cell {i,j) itself). 



^Our language differs slightly from that of |20| . 
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Proposition 8.12. [2^ Let A e C„. Let ^ C A fee the smallest partition (in 
the order by containment) such that the hook length of every cell in X/ fi is less than 
n. Then A >— > 6(A) := (Ai — /ii, A2 — j-i2, ■ • • ) defines a bijection Cn — > Bn- 

Example 8.13. For n = 4 and A as in Example 18.51 we have fj. ~ (7, 4, 2, 1, 1) 
so that A I— > (3, 3, 2, 2, 1, 1, 1, 1), which is obtained by reading the sizes of the rows 
of the skew diagram A//i, which is pictured below. 



8.6. fc-conjugate 

Transposition defines an involution lu on the set of partitions, which is easily 
seen to restrict to an involution on the set C„ of n-cores. Denote by 
the induced involution on Bn- For k — n — 1, uj'^''^ is the fc-conjugate map of 20J. 

Example 8.14. Reading the column sizes of the skew shape in Example 18. 131 
we have w^"*) (3, 3, 2, 2, 1, 1, 1, 1) = (3, 2, 2, 1, 1, 1, 1, 1, 1, 1). 

8.7. Prom Grassmannian elements to bounded partitions 

Recall from Section 12.11 the definition of an inversion of u> G 5„. Let the 
code of w be the sequence code(w) = (ci, C2, . . . , c„) where Ci is the number of 
inversions of w of the form (i, j) for some j; see also [2] where the code is called the 
inversion table. For a Grassmannian permutation it is easy to see that the code is 
a weakly increasing sequence of nonnegative integers that starts with 0; reversing 
this sequence yields a partition with fewer than n parts. Applying the transpose 
to, we obtain an [n — l)-bounded partition. 

Proposition 8.15. The composite bijection b o c : B„ is given by 

w ^ w''"' (a;(c„, . . . , C2, ci)) where code(u') = (ci, C2, . . . , c„). This bijection sat- 
isfies i(w) = |6 o c{w)\. 

Proof. Bjorner and Brcnti "2^, Theorem 4.4] recursively construct a bijection 
from bounded partitions to Grassmannian affine permutations w, from which one 
obtains the code code (it;). This recursive construction is compatible with the con- 
struction of a core from a Grassmannian permutation as in Section 18.21 □ 

Example 8.16. Let n = 4 and w = [-7, -1,4, 14]. Then w has inversions (2, 5), 
(3,5), (3,6), (3,9), (4,5), (4,6), (4,7), (4,9), (4,10), (4,11), (4,13), (4,14), (4,17), 
(4, 21) so that code(w) = (0, 1, 3, 10). Now w(code(w)) = (3, 2, 2, 1, 1, 1, 1, 1, 1, 1) 
and by Example 18.141 we have uj^^\uj{coAe{w)) = (3, 3, 2, 2, 1, 1, 1, 1), which agrees 
with Example 18. 131 
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Strong and Weak Tableaux Using Cores 

We now specialize all constructions to the special case of Grassmannian ele- 
ments. By the previous chapter we may work instead with cores. 

9.1. Weak tableaux on cores are fc-tableaux 

Consider a weak tableau U = (W^i,W2,...) where inside(W^i) = Ui-i and 
outside(Wi) = Ui for all i, and Ui G 5° for all i > 0. Let ^ = c(inside(C/)) and 
A ~ c(outside([/)). The weak tableau U can be depicted as a usual tableau of shape 
A//i in which all cells of the skew shape c{ui) / c{ui-i) have been filled with the letter 
i for all i. Such a tableau is called a k-tableau j20| . The following characterization 
of fc-tableaux is given by Lapointe, Morse and Wachs in j23j . 

Lemma 9.1. k -tableaux are semistandard (increasing in rows and strictly in- 
creasing in columns). Conversely, if a chain of cores in (left) weak order defines a 
semistandard tableau then it comes from a weak tableau. 

Proof. If Sa is a cyclically decreasing permutation, then Sj is never applied 
after Sj-^-i in s^lj). This means that two cells are never added on top of each other 
in the same column and thus Sa-ij)/"/ is a horizontal strip. This shows the first 
statement of the lemma. 

Now suppose that 6 — c(w) and 7 = c{v) where w ^ v ^ are such that 
(5/7 is a horizontal strip of size m (meaning that the difference £{w) — £{v) is to). 
Theorem 56 of 20J then says that S = Si-^ ■ ■ ■ Sj,„(7), for some ii,. . . ,im that are 
all distinct. Furthermore, in the proof of Theorem 56, it is shown that im can be 
chosen as the residue of the southeasternmost cell cse in S/j. By induction on the 
size of 5/7, it suffices to show that Si„+i is never applied at some point after Si^ so 
that • • • Si^ is cyclically decreasing. Suppose this is the case. Then, since 5/7 is 
a horizontal strip, this means that there is a cell of residue im -|- 1 to the northwest 
of Cse without a cell above it, and thus that the extremal cell of residue i„i to its 
left is not at the end of its row. This is a contradiction to [201 Proposition 15(1)] 
which says that all extremal cells of residue to the northwest of cse are at the 
end of their row. □ 

Example 9.2. Let n = A and I = and consider the weak tableau defined by 
the length-additive factorization of the Grassmannian element 

W = (S1)(S2)(S3)(S0)(S3S2S1)(S0S3S2)(S0S3)(S1S0) 

of Example 18.11 into cyclically decreasing elements. The corresponding sequence of 
cores is () c (2) c (2, 2) c (5, 2, 2) c (8, 5, 2, 2) c (9, 6, 3, 2, 1) c (9, 6, 3, 3, 1, 1) c 
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(9, 6, 3, 3, 1, 1, 1) c (10, 7, 4, 3, 2, 1, 1, 1), which gives the fc-tableau 
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9.2. Strong tableaux on cores 

We assume that / — with I as in Section [^T^ Since id is Grassmannian, by 
Proposition l2.6l anv strong tableau with inner shape id, involves only Grassmannian 
elements of 5„. 

The strong order on Grassmannian permutations corresponds to containment 
of cores. 

Proposition 9.3 ([MllSQ]). Forv,w e fi", v <w if and only if c{v) C c(u)). 

Lemma 9.4. Let fi be a core and t^^s ^ Sn o, reflection with r < s. Then 

(1) ^ 1^ <^^d, only if dj. > d^ using the extended offset sequence d of ^. 

(2) tr^sfJ- 2> fJ. if and only if dr > dg and for all r < i < s, di ^ [dg, d^] • 

Proof. (2) follows from (1). To prove (1), for cores fi and A, by Proposition 
19.31 ^ < A if and only if for all z G Z, J2j<iiPjif') ~PjW) ^ 0- Therefore i^.sM > M 
if and only if pr+qnilj) > Pa+qnilj) for all g g Z. dr > dg implies that this holds for 
q = and (|8.19p implies it for all other q. □ 

The head of a ribbon is its southeastmost cell. 

Proposition 9.5. Let ^ < X be cores with i^.sM ~ and < r < s. Then 

(1) s — r < n. 

(2) Each connected component of X/ fi is a ribbon with s — r cells in diagonals 
of residue r,r + l,...,s— 1. 

(3) The components are translates of each other and their heads lie on "con- 
secutive" diagonals of residue s — 1. 

(4) The skew shape X/ fi has dr — ds components. 

Example 9.6. Let n = 4 and I ~ and consider the strong marked cover 
C ^ {w ^ u) where w = [-8,-3,6,15], u = [-8,-6,9,15], and = (-1,10). 
We have m{C) = w{i) = 5. The affine Grassmannian permutations w and u 
have associated cores — (11, 8, 5, 5, 3, 3, 1, 1, 1) and A = (11, 8, 7, 6, 5, 4, 3, 2, 1) and 
offsets d{^) — (—1, 1, 3, —3) and d{X) — (2, —2, 3, —3) respectively. Letting (r, s) be 
such that tr,sA* — have tr^gW = u = wtij, so that tr^s = wtijw~^ = tu,(i),tu(j). 

Thus we may take r = w{i) — 2 and s = w{j) = 5. Letting d — d{fi), the skew 
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shape A//X has d2 — d5 — 1 — (—2) components, each of size 5 — 2. 
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Proof of Proposition 19.51 AU of the assertions follow from the first and 
Lemma 19.41 The first assertion follows from Lemma I9.4t l2|) and (|8.19p . □ 

Proposition 9.7. Let C = {w — > u) be a marked strong cover with w^u G 
and write c{w) = fi and c{u) — X. Then the head of one of the ribbons forming the 
connected components of X/fi, is on the diagonal m[C) ~ 1. 



Proof. We apply Proposition 19.51 with r = w{i) and s — w{j) = m{C). Since 
the r-th bit is being exchanged with the s-th bit 1 in the strong cover, the head 
of one of the ribbons must lie on the previous diagonal s — 1. □ 



Example 9.8. In Example 19. 61 the head of one of the ribbons in A//i is marked 
with a *; it is in the diagonal m{C) — 1 = 5 — 1. 

In light of Proposition 19. 7[ a marked strong cover of cores ^ A is given 
by the disjoint union of ribbons comprising the skew shape A//z, together with a 
marking on the head of one of the ribbons. For a strong strip, the sequence of 
marked cells must have strictly increasing diagonal indices. Since each strong cover 
on cores is a skew partition shape, this is equivalent to saying that the marked 
heads must proceed weakly to the south and strictly to the east. 

We make the following convention for strong tableaux on cores. Let T = 
[Si, S2, ■ ■ ■) be a strong tableau going between the elements uq < ui < ... of 
S°. Let A(*) = c(u,), /i = c(inside(r)), and A = c(outside(T)). We depict T as 
a tableau of shape X/fi. The strong strip Si is depicted by the skew subtableau 
of shape X^'''^ / X^^~^\ We distinguish the cells of the strong covers in Si by placing 
the subscripted letter ij in a cell if it occurs in the skew shape corresponding to 
the j-th strong cover in Si. Finally, a mark * must be placed on the head of some 
ribbon in each strong cover, such that the diagonals of the heads increase in each 
strong strip. 

Example 9.9. Here is a strong tableau for n — 3. 
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It has three nonempty strong strips 5*1 , S'2 , >S'3 . Si consists of a single marked 
strong cover -— ^ (1). S2 also consists of a single marked strong cover (1) — — > 
(1, 1). ^3 is given by (1, 1) °4 (2, 1, 1) (3, 1, 1) (5,3, 1). To understand the 
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markings, we convert the 3-cores in S3 into elements of Sn, giving the sequence 
[— 1, 3, 4], [—2, 3, 5], [—2, 2, 6], [—2, 0, 8] in window notation. CaU these uq, ui,U2,U3. 
Then the marks of the three covers in S3 are given by uo(4) = 2, ui(2) = 3, 
and 1*2(5) = 5. This means that the heads of the marked ribbons in 6*3 occm" in 
diagonals 2 — 1,3 — 1,5—1, which indeed is the case. 

Example 9.10. We compute an example of Theorem 14.91 using cores. Let 
n = 3, 7' = 2 and w — S2Sq. Then we have the equality (in A^^^) 

h2{x)Weaks2so{^) = 2Weaks2Sj32SQ(x) +WeaksosiS2sa{x) + Weaksgg^sj^o 
The right hand side corresponds to the following four strong strips on cores. 



ll I2 II I2 



9.3. Monomial expansion of ^-dependent /c-Schur functions 

The fc-Schur functions were discovered by Lapointe, Lascoux and Morse when 
studying the Macdonald polynomials H\{x;q,t). To summarize, great attention 
has been given to the study of the q, t-Kostka coefficients in the Schur expansion, 

(9.1) Hf,{x;q,t)=J2Kx^iq,t)sx. 

A 

By way of geometry of Hilbert schemes, Haiman proved [9] that the K\^{q,t) lie 
in N[g, t] and are associated to the characters of irreducible representations in a 
bigraded representation of the symmetric group [7] . A long-standing open problem 
is to find a tableaux characterization for these expansion coefficients. For one thing, 
it is known that the number of monomial terms in q and t that occur in a given 
K\p,{q,t) equals the number of standard tableaux of shape A. Furthermore, in the 
case that g = (the Hall-Littlewood case), there is a beautiful solution to this 
problem given by the charge statistic on tableaux |26| . 

The fc-Schur functions arose in the study of this problem when it was empirically 
observed that there exists a remarkable refinement of the expansion (|9.ip . To be 
more precise, computer evidence suggested the existence of a family of polynomials 
[18j defined by certain sets of tableaux A^^ as: 

(9.2) sf{x-t)= J2 ^''"'■''^^^ «s/.ape(T) 

with the property that any Macdonald polynomial indexed by a fc-bounded partition 
A can be decomposed as: 

(9.3) H,ix;q,t)^ K^'Kl^t)s^^\x-t), (g, t) G N[g, . 

Moreover, given that in this setting s{^^ (x; t) — for large fc, such a decomposition 
reduces to (|9.ip when k is large enough. 
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The original definition of s^'^'' [x; t) given in |18j is a combinatorial characteri- 
zation of the sets A^^ . As it relies too heavily on tableaux combinatorics to warrant 
full presentation here, we will instead give a less cumbersome definition of s^f^\x; t) 
from [19] that is conjectured to equal s^^\x;t). 

Formally, for a fc-bounded partition A ~ (Ai, . . . , A^), the definition is simply 

(9.4) ~s'^^\x-t) ^Tiyi---Tl';^Bl-l. 

In this formula, the operators are the vertex operators introduced in that 
recursively build Hall-Littlewood polynomials: 

and the operators Tm \ as we will see, encode the complexity of fc-Schur functions. 
To define t}^\ we require the construction of an auxiliary basis {G\{x;t)^ . 
for Aj' (the Q-linear span of Hall-Littlewood polynomials indexed by fc-bounded 
partitions). This done, is defined by setting 

n \ (h\, , { c'^y^ (x\t) if Ai = m , 

(9.5) T^^Gf\x-t) = <^ ^ ^ ' ^ 

[0 otherwise . 

The basis g'"^^ (x; t) is best understood by working first under the specialization 
t = 1. Associate to any fc-bounded partition A a sequence of partitions, 

a-'^- = (a(i),a(2),...,aW), 

called the k-split of A . The sequence A^'^ is obtained by partitioning A (without 
rearranging the entries) into partitions A^*' with main hook-length equal to fc, for 
all i < r. The last partition in A^'^' may have main hook-length less than fc. For 
example, 

(3,2,2,2,1,1)^3 = ((3), (2, 2), (2,1), (1)) 

and 

(3,2,2,2,1,1)^4^ ((3, 2), (2,2,1), (1)). 
It is important to note that A^'' = (A) when the main hook-length of A is < fc. 
This given, let g'^^\x; 1) be the ordinary Schur function product 

G'i'''(2;;l) = S^iDSxm-'-SxM ■ (1.11) 

These functions are called "k-split" polynomials. 

The fc-split polynomials when t 1 are a subfamily of a family of polynomials, 
Hs{x;t), known as generalized Kostka polynomials (see for instance j35|, ,36 ). 
These polynomials, indexed by arbitrary sequences S of partitions, are ^-analogs of 
products of Schur functions. The fc-split polynomials correspond to the generalized 
Kostka polynomials for which S — A^'^ . That is. 

The primary goal of investigations carried out in |18|, I19j was to find a fruitful 
characterization of fc-Schur functions. The empirical evidence that these functions 
satisfy analogs of Schur function properties such as Fieri and Littlewood- Richardson 
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rules at t — 1 motivated an independent study of the s)i (x) — sj^ (x; 1) case. 
The intricacy of the original definitions prompted a third (conjecturally equivalent) 
characterization for the special case that t — 1 in [21] . That is the definition in 
terms of weak fc-tableaux that we have used for sl!!^\x) throughout this article. 

The developments in this article have led us to prove in Theorem I4.f f I that 
these ^ (x) can be defined by the weight generating function of strong fc-tableaux, 
providing a striking refinement of the classical combinatorial definition of Schur 
functions. Although our work here concerns only the special case when t = 1, the 
family of strong fc-tableaux also seems to play a natural role in the general theory 
of fc-Schur functions. 

Proposition l9 . Sl allows us to give a conjectural monomial expansion for s^'^^ (x; t). 

Suppose C — fi A is a marked strong cover of fc -I- 1-cores. By Proposition 19.51 
the skew shape A//i consists of m identical ribbons each of height h, where the 
height of a ribbon is the number of rows it occupies. Define the spin of C to be 
spin(C) = m{h — 1) + {I — 1) where the marked ribbon in C is the l-th one from 
the top. The spin(T) of a strong tableau T is the sum of the spins of its marked 
strong covers. For example, the strong tableau of Example 19.91 has spin 2 = 0-1-0 
+ f + + f . Our use of the name "spin" is due to the similarities between this 
statistic and Lascoux, Leclerc and Thibon's spin statistic for ribbon tableaux [25j . 

Conjecture 9.fl. Let A e Z?„ where n = fc + 1. The k-Schur functions 
of |18L I19j are such that 

s[^\x;t) = Sf\x;t) = ^x^'W) i^P-(«) , 
Q 

where the summation runs over strong tableaux Q of shape given by the n-core 
6-i(A). 

The conjecture has been tested for all fc and all fc-bounded partitions up to 
degree 10. We should note that this conjecture is still open in the case t = 1, since 
the fc-Schur functions at t = 1 that we use in this article are those of |21|, which 
are only conjectured to coincide with the special case f = 1 of the fc-Schur functions 
of pi [19] . 

9.4. Enumeration of standard strong and weak tableaux 

A strong (or weak) tableau of shape vu/v is standard if its weight is the com- 
position (1, 1, . . . , 1, 0, . . .) where the number of I's is equal to £{w) — i{v). We will 
now briefly discuss the enumeration of standard strong and weak tableaux for small 
values of n. This will also serve as examples for the material of this section. Unfor- 
tunately, we have not been able to find elegant formulae, similar to the hook-length 
formula, for these numbers in general. 

For w,v G Sn, let /s'J'/o„g (or /^/^k) denote the (finite) number of standard 
strong (or weak) tableaux of shape w/v. As usual when v — id, we will just write 
/strong (oi' /weak)- ^hc affiue insertion bijection of Theorem 14.21 (with I — 0) has 
the following immediate consequence when restricted to permutation matrices: for 
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each m £ Z>o we have 

(9.6) E /strong /weak- 

i{w)—m 

Note that we have used the fact that f^^-oag = if w ^ S'j'J. 

Remark 9.12. The number /^^ak ^' ^ '^'n is equal to the number of reduced 
words of w. They were studied to some extent in [151 120j . It foUows from |15| that 
for each w G S'n we have a formula 

/weak = ^wvf v/eak 

e(v)=e{w) 

for some integers a^v G 2 arising from geometry. It is known from |16L I32j that 
Qwv are in fact positive. 

9.4.1. Case n — 2. Let n = 2. For each m G Z>o there is a unique w = 
w,n € S','^, which has a unique reduced word of the form w = • ■ ■ sqSiSo- The core 
c{'Wm) is the staircase {m,m — 1, . . . , 1) which has bounded partition (1™). There 
is a unique weak tableau with shape c{wm), of the form 



4 








3 


4 






2 


3 


4 




1 


2 


3 


4 



Thus (|9.6p says that m! = /g^^ng. Indeed, up to marking, there is only one standard 
strong tableaux with shape Wm, and each of the skew shapes c{wi) / c(wi-i) is a 
disjoint union of i squares. Thus there are a total of m! choices for the markings. 
Note that there is no condition on the markings to be increasing, since all strong 
strips are of size 1 (or in other words, all strong covers belong to different strong 
strips). 

9.4.2. Case n = 3. Let n = 3 and m E Z>o. There are lm/2\ + 1 Grassman- 
nian elements with length m. We denote by Wm,i € S'JJ the element with bounded 
partition {b o c){wm.i) = (2^ l'"-^'^) where £ £ [0, [m/2\]. Thus c{wm.o) is "tall" 
and c{w„i,im/2\) is "wide". 

For example when m = 3, we have Wm,o = siS2So and Wm,i = S2SiSq. The 
weak order is described in the following result, which has a straightforward proof. 

Proposition 9.13. Let n — 3. If m is even then the weak covers w < v with 
£{w) = m are given completely by w^^ i -< Wm+i,i- If m is odd then the weak covers 
w ^ V with £{w) = TO are given completely by Wm,e -< Wm+i.i and Wm.i -< w„i+i/+i. 

Remark 9.14. The Hasse diagram of the weak order of is in fact (part of) 
a honeycomb lattice (see |201 Figure 1] ) which had earlier appeared in related work 
of Fomin [4] where it was called "Hex" . 
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9. STRONG AND WEAK TABLEAUX USING CORES 



It follows immediately from Proposition 19. 131 that 



' i.Wm.-l,i 

•I weak 




if m 


J J weak 


' •> weak 


if m 


J weak 




if m 


^ weak 


/2~1 


if m 



weak i n is even and £ = 0, 



Solving these recursions, one obtains 

Proposition 9.15. The numbers of standard weak tableaux for n — 3 are given 



by 

_ ,'LW2J 

/ weak 



£ 

In particular the total number of weak tableaux with Grassmannian shape of length 
m is given by 

Erw ^ r,L™/2j 
J weak 

£{w)—m 

Now we turn to the calculation of /g'trong- Rather surprisingly, we find that 
/strong depends only on m. We first describe the strong covers. 

Proposition 9.16. Let n — 3. Suppose m is even. Then we have Wm,£ < 
Wrn+i,t, Wrn,i < Wm+i,e+i and Wm.i < Wm+ii-i whenever these permutations exist. 
The corresponding difference of cores have to/2+1, £ + I, and m/2 — £ + I compo- 
nents respectively. Suppose m is odd. Then Wm.i < Wm+i,i and Wm.i < Wm+i,£+i- 
The corresponding difference of cores have [to/2J — ^ + 1 and £ + 1 components 
respectively. Furthermore this describes all the (marked) strong covers in S^. 

Note that in particular if £{w) is odd then w < v if and only ii w ^ v and 

eiv) ^ £{w) + 1. 



Proof. To prove the result, we use the language of cores and Proposition 
which says that strong order corresponds to inclusion of cores. The width of a 
partition A is its first part Ai and its height is equal to the number of parts. We 
first note that the 3-core c{wm,i) has height m — £ and width m — ( [to/2J — £). This 
observation and Proposition 19.31 is enough to show that the stated strong covers 
are the only ones possible. To show that the stated pairs are indeed strong covers, 
we use the "/c-skew" shape of 1201 , which is denoted A//i in Proposition 18. 121 The 
/c-skew shapes of Wm,£ (which have outside shape c{wm,£)) are of the form 



for TO even, and 




9.4. ENUMERATION OF STANDARD STRONG AND WEAK TABLEAUX 
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□ 



for m odd. In both cases, one has ^-horizontal dominoes and \rn/2\ — £ vertical 
dominoes. With this description it is easy to see the pairs stated in the proposition 
are indeed strong covers with the stated number of possible markings. 

Using Proposition 19.161 one obtains the following recursions for /strong- 

/strong /strong {Tfl j Z I) /strong 



for m even, and 

^strong "^./strong ^ I '"'/ ^ I J strong 

for m odd. By induction one easily establishes 



(Lm/2J -t)f, 



strong 



Proposition 9.17. Let n = 3. The numbers /strong of standard strong tableaux 
depend only on m and are given by 

J si 



/strong 2l™/2l ' 

This agrees readily with Proposition 19.151 and (|9.6p . 

Example 9.18. Suppose to = 4. Then there are three Grassmannian elements 
W4,o, W4^i, W4^2 which have cores (2211), (311), (42) respectively. We have the follow- 
ing weak and strong tableaux, illustrating the identity (|9.6p : 24 = l- 6 + 2- 6 + l-6. 
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CHAPTER 10 



AfRne Insertion in Terms of Cores 

We now translate the afRne insertion algorithm for the special case of Grass- 
mannian elements, into the language of cores. By the proof of Theorem 14.21 it 
suffices to describe the local rule of Chapter [5] and its reverse in Chapter [SI in 
terms of cores. 

10.1. Internal insertion for cores 

Let C be a strong cover fji < tr^s ■ H = X with m(C) — s. By Proposition 19.71 
this is equivalent to marking the cell of A//i on the diagonal s — 1. We shall use 
this equivalence freely without further mention. We recall from Section 13.41 the 
definitions of initial pair, final pair, and commutation. The general definition for 
internal insertion is in Section [5. II 

We first express the condition for commutation in terms of cores. 

Let W = {fi he a. weak strip and the input final pair for the 

internal insertion at C, and let {W , S') be the output final pair, with outside(VF') — 
7- 

A' 

In every case we shall define W' — {X ^ j) in terms of a set A' C Z/nZ. 

Lemma 10.1. (W,C) does not commute if and only if the marked component 
of X/fi is contained in v/fJ.. Moreover, if this occurs then each component of X/fj, 
is contained within a single row. 

Proof. By Lemma [3.151 {W,C) does not commute if and only if the marked 
component of C is contained in the skew shape v/ ii and the tail (northwestmost cell) 
of the marked component has residue equal to the minimum of some cyclic interval 
/ in A. Since v/ fi corresponds to a weak strip, by Lemma I^TT] it is a horizontal strip 
in the usual sense. This implies that each of the ribbon components of A//x, lies 
in a single row. In particular the above condition regarding the tail, automatically 
holds. □ 

Using Lemma [10. II we rephrase Cases A, B, and C of internal insertion (which 
occur in Sections 17.21 17.31 and 17.41 respectively) in terms of cores. 

10.1.1. Commuting case for cores. 

Case A (Commuting case) Suppose the marked component of A//x is not con- 
tained in v/fJ.. Then let A' = A and S' = S[U C where C" is the strong cover 
v<-f = ic^(r),c^(s) • 1^, with m(C") = ca{s). 
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10.1.2. Noncommuting cases for cores. Let po be the diagonal of the head 
of the marked component of C and = A\{pq}. 

Case B (Normal bumping case) Suppose the marked component of A//i is con- 
tained in \/u and that either size(S'() = 0, or size(S'( ) > and m(C) ^ m(last(S'( )). 
Let go be the diagonal of the tail of the marked component of A//i. Let q < phe 
the unique pair of consecutive A^-nice integers such that q < qq, X has an addable 
cell in the diagonal q, and q is maximal. Let A' = U {p — 1} and S" = 5( U C 
where C is the strong cover u <^ :— tcj^(q)^CA{p) ' ^ with m{C') = ca{p)- 
Case C (Replacement Bump) Suppose the marked component of A//i is con- 
tained in \/v, sizc(S'() > 0, and m{C) — m(last(S'()). Let v~ be the core obtained 
by removing last(S'j^) from v. Let q < phe the unique pair of consecutive A^-bad 
integers such that q < po, v~ has an addable cell in the diagonal q, and q is maxi- 
mal. Set A' = A^\J {q} and 5" = (5;\last(50) U C'~ U C where C'~ is the strong 
cover < v' tq,p ■ with m{C' ) = p, and C" is the strong cover v' < 7, with 
to(C") = t,.p m(C). 

10.2. External Insertion for cores (Case X) 

Recall the general case of external insertion given in Subsection 15.1.31 Let 
W = (ii v) he a, weak strip. Let q — vi and p > q the next larger A-bad integer. 

Let A' = AU {q} and W' — (/i ^ 7). Let C' be the strong cover h' < j :— tq^p v 
with m(C') = p. This marks the component of "f /h' in the first row. 

External insertion on {W,S[) is given by computing {W',C') as above and 
setting S' = S[UC'. 



10.3. An example 

3 and let 



Example 10.2. Let n 



The growth diagram to compute the image (P, Q) of m is given by Figure 110.21 
Each row of the diagram defines a strong tableau, which is depicted to the right of 
the row. The tableau Q is given by 






3 








Q = 


2 


3 


3 






1 


2 


2 


3|3| 



We explain the computation of the last row of 2 x 2 squares. The first square 
consists of a single external insertion where the input data consists of empty weak 
and strong strips, all going from to itself. This external insertion just adds the 
residue 0, giving the new shape (1). 

The second square has input strong strip S that consists of a single marked 
strong cover C = (0 < (1) = to, 1(0)) with m{C) = 1, and input weak strip 
W — {0 (1)) with A — {0}. Since the marked component of C consists of the 
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32 




2J 


31 


3*2 



21 



31 



33 33 



Figure 1 . Growth diagram 



cell (1, 1) which is contained in W, we are in the noncommuting case. Since the 

old output strip S[ is empty, Case B holds. We have pn = qo = 0, q = —1, and 

p = 0, so that A' = { — 1} = {2} with output weak strip (1) ^ (1,1) and the output 
strong strip consists of a single cover C" = ((1) < (1, 1)) with m(C") = 0. 

The third square has the input strong strip 5 = ((1) < (2) < (3, 1)) = Ci U C2 
with m(Ci) = 2 and m(C2) = 3. It has input weak strip W = ((1) (1, 1)) where 
A = {2}. The internal insertion at Ci is in Case A: the marked component consists 
of the cell (1, 2), which is not contained in W, which consists of the cell (2, 1). This 
insertion produces the output strong strip (1, 1) < (3, 1) with mark 3 and the same 
set A, which now is associated with the weak strip (2) ~^ (3, 1). 

Next we perform the internal insertion at C2. We reindex with input weak strip 
W = ((2) ^ (3, 1)), and A = {2}, old output strong strip S{ = ((1, 1) < (3, 1)), 
and C = Ci = ((2) < (3, 1)) with m(C) = 3. The marked component of C is the 
single cell (1,3) which is contained in W. This is the noncommuting case. Now 
m(last(S'i)) = 3 = m(C) so we are in Case C. We havepo = 2, — {}, = (1, 1), 
g = 1, p = 2. Thus A' = {^} = {T}, with output weak strip ((3, 1) ^ (3, 1, 1) and 
output strong strip S' = ((1, 1) < (2, 1, 1) < (3, 1, 1)) = C( U Cj, where m(C() = 2 
and miC'o) = 3. 

To finish up we apply a single external insertion to the output final pair from 
the previous step, which arc rcindcxcd as T4^ = ((3, 1) (3, 1, 1)) with A = {1} 
and S = ((1, 1) < (2, 1, 1) < (3, 1, 1)) = C( U with m(CO = 2 and m(C^) 3. 

We have g = 3, p = 5, so that A' = {0, 1} and W = ((3, 1) ^ (5, 3, 1)) and 
C" = ((3, 1, 1) < (5, 3, 1)) with m(C") = 5 and S" = 5 U C". 
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10.4. Standard case 

The insertion algorithm becomes particularly simple in the "standard" case, by 
which we mean the restriction of the insertion bijection to the subset of permutation 
matrices. All weak strips that occur are empty or a single weak cover. All strong 
strips that occur are empty or single marked strong covers, and Case C never 
occurs. In particular, in the standard case, the insertion process is "context-free" ; 
each internal and external insertion is independent of previous computations. 

Let us adopt the notation of Section 110.11 for internal insertion. We have the 
weak cover W = {fi ^ SafJ, = v) (with singleton weak strip set A = {a}) and the 
strong strip given by a single marked strong cover S = < /it^s = A), marked at 
the cell on diagonal s — 1. 

Let us consider Lemma 110.11 for the standard case: it says that {W, C) does 
not commute if and only if the marked component is contained in W ^ which is a 
weak cover, whose components are single cells. Hence the components of the strong 
cover are single cells, and it follows that the strong cover and weak cover coincide. 
Therefore the noncommuting case is simply described by the conditions [i^ \ = 
and the marked component is the single cell of A//i in the diagonal s — 1 where 

fj, < \ = jltrs- 

So Case A occurs if and only \i \ ^ and the computation goes as before: 
the output weak cover is A ^ Sa • A = 7 and the output marked strong cover is 
1/ < 7 = vtrs with the cell on the same diagonal s — 1 marked. 

Case B occurs if and only if A = i^. In that case, one may show that A' — [q] 
where q is the diagonal of the first A-addable cell to the northwest of the marked 
cell. The marked cell for the output marked cover is the cell in diagonal q. 

Case C does not occur. 

For external insertion, we note that it will only occur if the input strong strip 
is empty. Therefore in this case, the output marked strong cover consists of the 
weak cover with new residue given by that of the /i-addable cell in the first row, 
with that cell in the first row marked. 



Example 10.3. Let n = 3 and let 
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0/ 



The growth diagram is given in Figure [10.31 A * in a partition diagram indicates 
the marking for the strong cover coming from its left. 
The P and Q tableaux are given by 
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■ 



■n*i- 



Figure 2. Growth diagram of a standard insertion 



10.5. Coincidence with RSK as n — > cxd 

Theorem 10.4. As n ^ oo the bijection of Theorem \4-3\ converges to RSK row 
insertion. 

Proof. In the limit the set of n-bounded matrices becomes the set of all 
matrices with finitely many nonzero entries. All residues of corner cells in a partition 
will be distinct. Thus a weak strip becomes an ordinary horizontal strip with one 
cell per residue and a weak tableau becomes a semistandard one. A marked strong 
cover becomes a skew shape consisting of a single cell, a strong strip becomes a 
horizontal strip, and a strong tableau becomes a semistandard one. 

We now consider the local rule. The data consists of a horizontal strip S = 
X/fi, a horizontal strip W — ly/n, and some e € Z>o. The algorithm consists of 
performing internal insertions on W at the cells of S from left to right. Case A 
occurs when the active cell C (the one at which the internal insertion occurs) is 
not in W, so no bumping takes place. Case B occurs when the active cell is in W, 
in which case the active cell C is removed from the "weak strip" and another cell 
C is added, namely, the next addable one of smaller residue. This is equivalent to 
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bumping the cell to the end of the next row. Case C never occurs because cells are 
always bumped to strictly earlier diagonals. 

Finally, each of the e external insertions is given by adding a cell to the end of 
the first row (and putting this cell in the new weak and strong strips). 

Since this coincides with the local rule for ordinary RSK row insertion, the 
Theorem follows. □ 

10.6. The bijection for n ~ 3 and m ~ A 

In Sect ion [9^ we described all the strong and weak tableaux for n = 3. Now we 
give the entire (standard) bijection for m = 4 in this case. For convenience, we have 
included the outcome of the usual Robinson-Schensted bijection for comparison. In 
the following table, the strong tableau is always drawn first. 



10.6. THE BIJECTION 



FOR 71 = 3 AND m = 4 
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n = 3 
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